The best (and my favorite) solution for both very high security and ease of use is a hardware bitcoin wallet which protects your private keys like the Trezor.
Trezor is a small USB-key sized device with a small screen and two buttons which connects to your computer (Win, Mac & Linux) and signs bitcoin transactions for you. Your private keys to your bitcoins never leave the Trezor device.
During set-up you are presented a 12-24 word seed which you need to write down and keep in a safe place. The seed enables you to recover everything in case your Trezor gets lost or broken. After that you set a PIN which can be more than 4 digits and you can begin sending funds to Trezor using the Trezor-only web wallet at mytrezor.com. Integration with popular software wallets like electrum, multibit, armory, etc…
Main advantages of the Trezor hardware wallet:
- Very secure storage even if your main computer is compromised
- Easy one time backup: write down a 12-24 word seed and you are set, even if your Trezor gets destroyed.
You can find more information about Trezor at BuyTrezor.com.