Add one to a number using Emacs’ regexp

I recently had to add one to all numbers of the form "n" where n is a number in a text file. Thanks to jlf on #emacs, I could do so with the the command replace-regexp to change \"\([0-9]+\)\" to "\,(1+ \#1)". I'm never going to remember this so I'm writing it down now.