H264 USB Camera on Raspberry4-server, ubuntu client

Submitted by maphstra on #1

Hey, we are using a raspberry 4 to share our USB-camera. This works fine on a windows client, but on a ubuntu 18.04 client the cameras are not working.
Using cheese, I can select the camera but get a black screen, using vlc or ffplay i get an image with a LOT of noise, unable to recognize anything there.
Here are some errors from vlc:

$ vlc v4l2:///dev/video4
VLC media player 3.0.8 Vetinari (revision 3.0.8-0-gf350b6b5a7)
[000056465f5f1630] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
[00007fc974009a20] avcodec decoder: Using Intel i965 driver for Intel(R) Coffee Lake - 2.1.0 for hardware decoding
[h264 @ 0x7fc97403c3c0] No start code is found.
[h264 @ 0x7fc97403c3c0] Error splitting the input into NAL units.
[00007fc974009a20] main decoder error: buffer deadlock prevented
[h264 @ 0x7fc97403c3c0] No start code is found.
[h264 @ 0x7fc97403c3c0] Error splitting the input into NAL units.
[h264 @ 0x7fc9740585a0] No start code is found.
[h264 @ 0x7fc9740585a0] Error splitting the input into NAL units.

these errors keep repeating.

Errors from ffplay:

$ ffplay /dev/video4
ffplay version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2003-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
[h264 @ 0x7f5a100020e0] concealing 6980 DC, 6980 AC, 6980 MV errors in I frame
Input #0, video4linux2,v4l2, from '/dev/video4':B sq= 0B f=0/0
Duration: N/A, start: 6097.172441, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 30 fps, 30 tbr, 1000k tbn, 2000k tbc
[h264 @ 0x7f5a100529a0] concealing 6980 DC, 6980 AC, 6980 MV errors in I frame
[h264 @ 0x7f5a100e5540] concealing 2347 DC, 2347 AC, 2347 MV errors in P frame
[h264 @ 0x7f5a101018c0] concealing 2870 DC, 2870 AC, 2870 MV errors in P frame
[h264 @ 0x7f5a1011dc40] concealing 2787 DC, 2787 AC, 2787 MV errors in P frame
[h264 @ 0x7f5a10139fc0] concealing 6674 DC, 6674 AC, 6674 MV errors in P frame

These errors also keep repeating.

Do you have any idea whats going wrong here?
Regards,
Mads

Submitted by Michael on Thu, 2019/11/21 - 02:37

#2

OK i have an idea, the USB memory buffer parameter on the pi may be too small

So on the pi server

Stop the vhusbd process (pkill vhusbd)

Run 

echo 128 > /sys/module/usbcore/parameters/usbfs_memory_mb

then start the virtualhere server process again.

Also, please update your server to the latest 4.0.6 and client to 4.9.4 for your setups

Submitted by wunnfa on Fri, 2021/05/07 - 01:24

#3

Hello Michael,

I have the same Problem on UBUNTU 20.04 LTS and a PI 3B. I tried to increase the USB buffer size with:

sudo sh -c 'echo 128 > /sys/module/usbcore/parameters/usbfs_memory_mb'

since your reply promted a "permission denied"

Still, my Video Camera shows a black screen using Cheese.

Has this been resolved somehwere else?

I use the default server version and want to see that every device I use works with my Ubuntu and Windows machine. If so I will gladly buy a license.

Submitted by Michael on Fri, 2021/05/07 - 07:45

#4

Works fine for me on 20.10 using a Logitech C930e (using a pi4)

Note that if it says permission denied, then it didnt make the change because you dont have permission :)

So do this

sudo -s

then do this

echo 128 > /sys/module/usbcore/parameters/usbfs_memory_mb

and turn down the resolution of the camera capture also, as it might be too much data for the pi3 to handle. Note: you really should be using a pi4 if you want to pass through video/audio from a webcam because the pi3 only has one usb port for everything including network, whereas the pi4 has a true usb 3 and 2 ports which dont share bandwidth with the network interface

Submitted by wunnfa on Fri, 2021/05/07 - 23:30

#5

Thank you for your fast reply.

I was able to verify that the changes happened. I do not think its a USB Buffer Problem. Since I am really curious, I wanted to know why the usb-lan-camera connection works with my Windows Laptop and not with my Ubuntu Mini PC. Here is what I found out:

I monitored the TCP traffic using tcpdump on the server side and the following difference occured to me:

- When using the camera on the Windows machine many Data Pakets with the default MTU of 1500 bytes are transmitted to the Windows Machine. Sometimes the TCP Protocol sends a Push. I guess that is normal behaviour, since the amount of Data to transmit is high for a Webcam and the push comes when some valid Application Data has been transmitted.

- When using the camera on the Ubuntu machine I see a huge difference in TCP behaviour. After already every 8th TCP Paket, a TCP Push follows. I am very new to the topic, but a Push usually signals the receiving application that valid data can be assembled. In ~14 kB of data no single camera image will fit. This would at least explain why I see a black screen in some applications, and in others (like Zoom) I see only a few lines of the (valid) image.

It raises the question: Why is the TCP transmission from the PI to the Ubuntu Machine so inconsistent compared to the transmission to the Windows machine?

Submitted by Michael on Sat, 2021/05/08 - 09:14

#6

It doesnt have anything to do with tcp packets.

You should get a pi4 and try again..

Submitted by wunnfa on Sat, 2021/05/08 - 19:50

#7

Yes, I think I went down a rabbit hole there. Still, I do not quite get your recommendation: Why does the whole setup work with the Windows Client then?

I don't like the price-tag on the PI4, assuming I will also buy a license for virtualhere. Costs will be approx. 100 Euro for the whole setup. My ultimate goal is to get rid of all USB cables, so I think I will get rid of the PI idea and use the Mini-Barebone-Ubuntu-PC as server. The Ubuntu-PC is also the workstation we were talking about, so there won't be any problems with USB-LAN artifacts on the camera any more.

While I was testing my equipment with the PC server, I also found out that my USB-Wireless Headset does bring great sound-quality over the LAN connection, but the Microphone Recordings do not work. It seems to be a similar issue, since voice is recorded but with a lot of artifacts where the timeline of the recording has hickup. I will test it once again when every device is connected via 1G Lan, at the moment devices share the network over a 100M connection.

Cheers

Submitted by Michael on Sun, 2021/05/09 - 11:32

#8

The windows client probably uses a different webcam video mode than the linux client. Thats probably why

Submitted by wunnfa on Tue, 2021/05/11 - 09:26

#9

Hi Michael,

I am happy with my Setup so far. With a 1G LAN and using the Mini-Barebone-PC (Ryzen 3) I can share the USB Camera as well as the Headset with another Windows Machine. I even got rid of the microphone recording artifacts (choppy recordings) using VoiceMeeter.

Now a new Problem arises: Even though my Microphone seems to work perfectly now, I can hear scratches in the playback of any source like Spotify. If I close VoiceMeter and use the Headset Sound as default, no scratches occur, but then my microphone will not work again.

Do you have any clues on how to deal with these scratches (Using VoiceMeter)?

Thank you

Submitted by Michael on Tue, 2021/05/11 - 10:01

#10

Im not sure actually, but my first guess would be to also use voicemeeter for the headphones (as well as the microphone)