All of the side pieces are the same. All of the lateral pieces wont work because teh O-360 cyls are longer than the O-320 (the O-360 engine is 1 inch wider than the O-320) You could probably make O-360 baffle parts work on an O-320, but I don't think it can work the other way around.
