You can use whichever USB port is empty and available for your USB memory stick.  However if none are available, you will need to either unplug another device or possibly get a USB hub to allow for more devices to be connected to your computer.

If you are using Windows XP or Vista, when you connect your USB stick to your computer, windows will automatically detect the hardware and install the necessary software.  Once the software has been installed, you will now see the USB stick listed as an extra drive letter when your explore the contents of your C: drive.  It will most likely have the default name of Kingston, Lexicon or whichever brand of memory stick you purchased.

At this point, you treat the memory stick just as if it was another folder on your computer.  Simply cut and paste, or copy photos from your C: drive to your USB memory stick drive name.  When you are done, select the eject hardware icon in your taskbar, and remove the memory stick when you are told it is safe to do so.