Thanks a lot for the information!
Very interesting project!
Yeah, fully concur on the head tracking aspect, at least if one aims for fully dynamic / real angular relative positioning.
I recently took part in a concept demo based on the latest Microsoft Hololens at work (ATC). Totally useless in the current state. And it will likely take years... It's not "just around the corner" yet. Likely a similar story with that latest Apple gadget that was just announced.
Nonetheless, from the flightsim arena, there are interesting and "working" methods for headtracking, even if they are only used more in the direction of interactive pointing/panning & controlling the FOV, somewhat like cursor. Helps the immersion factor and is quite intuitive, could be a first step.
Here in Europe, you or STRATUX would have to support the FLARM traffic protocol, as that is very widespread. The RS232 protocol is also quite well defined and documented:
https://www.flarm.com/wp-content/uploads/man/FTD-012-Data-Port-Interface-Control-Document-ICD.pdf
specifically the $PFLAA sentence is of interest.
The stream also always contains a high quality GPS source.
Regarding video overlay:
I've played with three approaches so far:
- OpenCV
https://opencv.org/ way too heavy for the kind o thing you want to achieve. That would become relevant if you want to autotrack things etc...
Haven't gone deep so far, was toying with the idea of object tracking, but got stuck on much more basic things such as reliably moving the camera, never made it to the autotracking, but did run first test ;-)
- If you have very low speed video / individual frames / mjpg for things like a webcam, i've had success with imagemagick, respectively mjpg-streamer.
-
Picamera is what you will want to use on the RPi4, as that has the most efficient overlay handling and i believe has about the right level of capability.
https://hackaday.io/project/12450-r...41957-getting-overlay-to-work-while-recording
is one similar sample use case. Doesn't cost too much performance.
looking forward to further updates!
good luck!