Be gentle but honest
I think the idea of an EAA tech councelor visit is a really good one, so that he hears from someone else besides you. That does put a tech councelor in a tough spot going in, knowing that he is going to have to tactfully critique lots of bad work -- maybe they get enough experience at that that it is not a burden, I don't know. (My hunch is that most work they see is pretty good, just need minor tips?)
A good friendship should be durable enough to withstand an honest discussion about this. If you can, keep it focused on the objects and not the person. Personal criticism will not help, but fair, honest critique of the work should be respected. It might be hard if someone has a lot of pride in their work and hears that it is deficient. Anyone would react defensively, and you will have to find a way to side-step that. I don't know, but maybe it is not so hard to hear it when there is an apparent lack of pride in workmanship.
Find a way to mix some positive comments in - and/or have proposed courses of action to recover (like put a doubler on the back side of the spar web where the bad z-bracket holes are).
Anyway, I do think you owe it to him and any potential future passengers and potential future owners to find a way to tell him that there is enough poor work here that the safety of the end result is in question. There are lingering potential liability issues for him in selling something that is built in a way that counters the direct instructions from the kit supplier.
It is easy for people to dismiss some poor work by saying there is lots of extra margin in the structure to tolerate a few defects, and that is more or less true. But enough defects can add up to where the structure is compromised.
It sounds like he is fairly casual about loaning his airplanes around, but if it comes to it, you could offer some other in-kind effort or reimbursement if needed to separate the airplane usage discussion from the construction issues. (hangar storage of the parts probably counts for a fair bit).
Number one suggestion: find a way to deal with it so it doesn't eat you up inside and cost you sleep. Its already costing you build time and attention on your own project, and that is already a high price. Resolve it, don't let it fester.
Hope there is some nugget of helpful ideas here for you,
Best,