To get started my preference is to use RubyInstaller for Windows. Both 1.8.x and 1.9.x variants are available. I group together my Ruby variants under \Programs\rubies.
Having installed multiple variants you can register and manage them using Pik. I recommend not changing the installation path from the default C:\pik. It is in fact possible to install Ruby using Pik (by default Pik installs rubies in %HOMEPATH%\.pik\rubies), but I prefer not to do so.
pik add d:\programs\rubies\ruby-1.8.7-p334\bin pik add d:\programs\rubies\ruby-1.9.2-p180\bin
To access code repositories you will need to download and install msysGit for Windows. I also strongly recommend TortoiseGit instead of Git GUI.
To build native extension gems like Nokogiri you will need DevKit. I tried
pik package devkit install
but this did not work for me (got HTTP 404) so I had to manually download and extract the DevKit. DevKit also includes MSYS/MinGW.
For an IDE I use Aptana Studio 3. Studio looks for an existing Git installation or installs its own private copy of portable Git. Studio then leverages the msysGit bash shell.
RubyInstaller includes gem, which can then be used to install bundler to complete the setup of the environment.