RPI4 with dual UVC cameras (512x640) has erratic image

Hello Michael,

We have a setup with two UVC compliant cameras (512x640 rgb) (FLIR BOSON ir cameras) attached to the same Pi4. The cameras are read-out sequentially, VID0, VID1, VID0, VID1,,, Obviously we use the licensed version and it is started like this:
sudo ./vhusbdarmpi4
Both client (Win10/64b) and server are the latest from the website as of 2021-02-20

We see an erratic behavior in the way that
1) sometimes an image is black,
2) sometimes one image is replicated to the other cameras display - giving two identical displays.

These events are intermittent and stay for 1 or few frames.

We have had this setup to work with older VH versions.

Have you any suggestions as to what we could try to go around this?

br henning

#2

FYI,
I tried with the generic server version vhusbdarm and it has similar problem, Br,

#3

I doubt virtualhere has a bug in this part. Its probably either a memory issue or a kernel bug.

Its not possible for virtualhere to mix the data from a usb device with another unless there is a bug in the linux kernel.

Anyway try this regarding the memory issue:

https://www.flir.com.au/support-center/iis/machine-vision/application-n…

If that doesnt work i would suggest trying to get the very latest kernel for the pi4

#4

Ok thanks.
I have made a few more tests: It is a fact that
1) system does not fail when going direct via usb (no VH)
2) system does not fail if streaming the video's in two different processes using two VLC instances (with VH)
3) System fails when using our custom client application (win10/64) using open-cv and python.

This makes me suspect that it could be more like a problem in the client or of course in open-cv/python and not in the server (PI4).

#5

It sounds like there is a race condition in your python client...

#6

thanks
I have discovered that there are network problems. I see a lot of: [Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)]
When I use a direct cable it works much better and I have not seen this error yet. So for the moment I am happy because anyway the connection server client is in any case going to be a direct cable connection.

It is tempting to explain the mixing of frames we see to be due to this networks error but tcp is supposed to be taking care of this. Not sure if it gives sense to you. If not this may be classified as thing to take note of and not necessarily understand.

There are still another issue which is unclear yet: The camera has a built-in serial port in addition to the UVC. The serial port gives problems. I have still to analyse this further. It behaves different with and without VH.

br henning
Frame 82384: 1514 bytes on wire (12112 bits), 1514 bytes captured (12112 bits) on interface \Device\NPF_{85EF5ADF-BE95-449E-BA8E-024DE4895AE1}, id 0
Ethernet II, Src: Raspberr_0b:2c:d5 (dc:a6:32:0b:2c:d5), Dst: IntelCor_7c:48:a4 (04:d3:b0:7c:48:a4)
Internet Protocol Version 4, Src: 192.168.50.239, Dst: 192.168.50.208
Transmission Control Protocol, Src Port: 7575, Dst Port: 52571, Seq: 98450721, Ack: 19726, Len: 1460
[Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)]

#7

TCP is rock solid because it will handle network errors/retries and will never duplicate data. TCP has been around for a very long time and all bugs have been ironed out on all OS's and hardware.

Ive never seen that error before and dont quite know what it means. I did a quick check on google and found this https://serverfault.com/questions/633866/http-not-finishing-over-lan-ha…

You might have to turn off power-saving on the serial device. VirtualHere wont support remote power saving of USB devices. E.g see here for a similar (networking device) https://www.virtualhere.com/content/virtualhere-drops-connection-after-…

#8

I have concluded
[Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)]
was a problem with an ethernet cable. At 1000Mb it fails and at 100Mb it works. It is not conclusive where the error is introduced. If not caused by VH it can be the client application (opencv/python/win10/64b) which does not handle errors properly. I will leave this as it is and not investigate further.

I will however continue to investigate the other problem I briefly mentioned: why the serial communication to the camera' com port is erratic now and then in particular at the beginning. It seems to be unrelated to the network issue but it does not occur with direct USB ie. no VH.