Sure you can, the time counts towards the "on the job experience" path to getting your A&P. But it's not exactly a walk in the park to do either: you need 1500 hours minimum (which the FAA says is equal to 3 years experience, go figure).
Of those 1500 hours you need to show experience in the various areas where A&Ps need to have knowledge and experience: both airframes AND powerplants. Hanging a brand new Lycoming on your RV does not fulfill the requirement to have powerplant experience. You need to be actually doing maintainence, which depending on what your local FSDO requires may be much more indepth.
Also, start keeping an A&P log NOW on all the work you do, buy a real A&P Log and write down as much as you can. If you ever do work with someone who is already an A&P, get them to signoff the work you did (in your A&P log) to show the FAA you are actually doing work and not making things up. This is one area that is pretty easy to fake, so to get your FSDO to accept your experience is not as simple as showing up with all the paperwork filled out.
I'm a Light Sport Repairment/Maintainence and this is the way that I'm working towards my certificate, so I'm already on the path. I know other guys who've done it this way and it IS doable, but you have to do it properly.
Jeff