Here are directions and a printable template for making a paper dreidel that can be assembled with glue and a pencil or a straw for the handle. A nice variation is the paper droidel. If you want all the materials pre-packaged you can purchase a kit.
For oragami fans, here are step by step instructions for making a folded paper dreidel with accompanying photographs that illustrate the process.
Marshmallow dreidels are a creative and yummy project - and while they don't spin terribly well - any activity with icing and chocolate kisses is ok in my book.
If you are looking for a more traditional wood dreidel, dreidelmaker.com has a nice set of parts that you can assemble on your own. You can also purchase blank wood dreidels from bayerwood.com and nvo.com. Hebrew letter rubber stamps can be used to mark the sides.
And if you really want to follow the song, here are instructions to make a little dreidel out of clay.