Speed is affected by a couple of different factors. You mentioned the
easiest to upgrade, so I will expand on those.
Memory essentially comes in two forms, RAM and virtual memory. You
need to find out a bit of information about your computer before you
go out and buy new RAM (probably around $100-$150). A quick, online
way to get this information is
http://www.crucial.com. The system
scan their can also point you towards buying RAM from them. You
should check other retailers for the same type of RAM at a lower price
before buying, though. Virtual memory is a little easier to expand,
but does not greatly increase performance (but it's free). Here is an
article on how to increase virtual memory
http://www.howtofixcomputers.com/guide/Virtual_Memory
Free hard drive space also affects speed. Before buying a bigger
drive, you may want to first clean some un-needed files (be carefull
to not delete system files) and defragement your drive (back up
first!). If you do end up buying a new hard drive, there are
advantages and disadvantages to both internal and external drives.
What version of Windows (or other) are you running?
Stevo