Easier to configure than I thought it would be. The defacto document on this lists a series of convoluted steps to get UTF-8 working.  All you really need to do is as follows:

1. Launch the local configurator via # dpkg-reconfigure locales
2. Select the locales you would like to support. I need Japanese support so I selected ja_JP.EUC, ja_JP.UTF-8, and en_US.UTF-8 in addition to already available locales. Click “Ok”.
3. Choose the default locale from the list; make sure it is a UTF-8 locale. Click “Ok”.
4. Login to another shell.

That’s it!

# locale charmap should now return “UTF-8”.