I upgraded a couple of MacPros (5,1) from 10.9 to 10.13.6
The linux server was upgraded to 4.0.3, the Mac client version now is 4.9.0
I'm serving licence dongles, primarily iLoks. They work for a few minutes before the Virtualhere client crashes, sometimes followed by a kernel panic.
I completely removed and reinstalled the kernel modules with reboots inbetween in order to make sure I have the right version installed.
Do you have any suggestions what to try to find the cause of the problem?
I attached logs that might help.
# kextstat |grep vh
19 0 0xffffff7f81007000 0x16000 0x16000 com.virtualhere.vhhcd (2.0.3) BF500686-DF47-3775-84D4-A81B41981813 <18 5 4 3>
Syslog:
Oct 6 20:23:49 101-MAC VirtualHere[1128]: VirtualHere Client crashed, backtrace follows...
Oct 6 20:23:49 101-MAC syslogd[58]: ASL Sender Statistics
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 0 VirtualHere 0x0000000108bdc1b3 _ZN16LocalApplication20OnFatalExceptionImplEv + 83
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 1 VirtualHere 0x000000010925957c _Z20wxFatalSignalHandleri + 28
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 2 libsystem_platform.dylib 0x00007fff5059ff5a _sigtramp + 26
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 3 ??? 0x0000000000000000 0x0 + 0
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 4 VirtualHere 0x000000010925eb0b _ZN12wxEvtHandler19ProcessEventLocallyER7wxEvent + 59
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 5 VirtualHere 0x000000010925ea1a _ZN12wxEvtHandler12ProcessEventER7wxEvent + 186
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 6 VirtualHere 0x000000010925e7b8 _ZN12wxEvtHandler20ProcessPendingEventsEv + 744
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 7 VirtualHere 0x000000010917f1ff _ZN16wxAppConsoleBase20ProcessPendingEventsEv + 287
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 8 VirtualHere 0x000000010923c232 _ZN13wxCFEventLoop29OSXCommonModeObserverCallBackEP19__CFRunLoopObserveriPv + 66
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 9 CoreFoundation 0x00007fff283c5737 __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 10 CoreFoundation 0x00007fff283c565f __CFRunLoopDoObservers + 511
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 11 CoreFoundation 0x00007fff283a8128 __CFRunLoopRun + 1240
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 12 CoreFoundation 0x00007fff283a79b7 CFRunLoopRunSpecific + 487
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 13 HIToolbox 0x00007fff27687d96 RunCurrentEventLoopInMode + 286
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 14 HIToolbox 0x00007fff27687a0f ReceiveNextEventCommon + 366
Oct 6 20:23:49 101-MAC VirtualHere[1128]: [bt]: 15 HIToolbox 0x00007fff27687884 _BlockUntilNextEventMatchingListInModeWithFilter + 64
Oct 6 20:23:49 101-MAC VirtualHere[1128]: VirtualHere Client backtrace complete
Crash report:
Anonymous UUID: B0FEB6C4-C1D6-BAD0-4949-0032EDA6E5F6
Sun Oct 6 22:11:10 2019
*** Panic Report ***
panic(cpu 0 caller 0xffffff8009e6e2e1): "USB-Chip[0x100000692]::terminate(kIOServiceSynchronous) timeout\n"@/BuildRoot/Library/Caches/com.apple.xbs/Sources/xnu/xnu-4570.71.2/iokit/Kernel/IOService.cpp:2297
Backtrace (CPU 0), Frame : Return Address
0xffffff8d08993a20 : 0xffffff800986c1c6
0xffffff8d08993a70 : 0xffffff8009995274
0xffffff8d08993ab0 : 0xffffff8009987544
0xffffff8d08993b20 : 0xffffff800981e1e0
0xffffff8d08993b40 : 0xffffff800986bc3c
0xffffff8d08993c70 : 0xffffff800986b9fc
0xffffff8d08993cd0 : 0xffffff8009e6e2e1
0xffffff8d08993d30 : 0xffffff8009e71e17
0xffffff8d08993db0 : 0xffffff7f8a597884
0xffffff8d08993dd0 : 0xffffff7f8b873f2b
0xffffff8d08993e10 : 0xffffff7f8b87a271
0xffffff8d08993e50 : 0xffffff7f8b87ae1d
0xffffff8d08993eb0 : 0xffffff7f8b872e99
0xffffff8d08993ed0 : 0xffffff80098a5844
0xffffff8d08993f40 : 0xffffff80098a53a5
0xffffff8d08993fa0 : 0xffffff800981d557
Kernel Extensions in backtrace:
com.apple.iokit.IOUSBFamily(900.4.1)[2837DD62-F2B0-3357-A164-D44FEF6810E0]@0xffffff7f8a55a000->0xffffff7f8a5f4fff
dependency: com.apple.iokit.IOPCIFamily(2.9)[AEA4C7F5-CCB0-338B-B471-CF28A9792522]@0xffffff7f8a094000
dependency: com.apple.iokit.IOUSBHostFamily(1.2)[CE61D704-9BFF-308E-BC1C-56FF071DBB74]@0xffffff7f8a4d9000
dependency: com.apple.driver.usb.AppleUSBCommon(1.0)[E7111D99-B893-3F96-9741-71C88BC6F66B]@0xffffff7f8a4d1000
com.apple.driver.AppleUSBLegacyHub(900.4.1)[F34036CF-9F86-3025-8C0B-18273A15F4A3]@0xffffff7f8b866000->0xffffff7f8b87efff
dependency: com.apple.iokit.IOUSBFamily(900.4.1)[2837DD62-F2B0-3357-A164-D44FEF6810E0]@0xffffff7f8a55a000
BSD process name corresponding to current thread: kernel_task
Mac OS version:
17G65
Kernel version:
Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64
Kernel UUID: 1AE5ACFD-3B6F-3D74-AD52-31F1430DBC6F
Kernel slide: 0x0000000009600000
Kernel text base: 0xffffff8009800000
__HIB text base: 0xffffff8009700000
System model name: MacPro5,1 (Mac-F221BEC8)
System uptime in nanoseconds: 9192510175795
last loaded kext at 9129183664629: com.apple.driver.usb.cdc 5.0.0 (addr 0xffffff7f8d059000, size 28672)
last unloaded kext at 391484943034: com.apple.driver.AppleUSBMergeNub 900.4.1 (addr 0xffffff7f8b898000, size 12288)
loaded kexts:
com.blackmagic-design.desktopvideo.iokit.framebufferdriver 10.6.1
com.RME.driver.HDSPMADI 4.13
com.blackmagic-design.driver.BlackmagicIO 10.6.1
com.virtualhere.vhhcd 2.0.3
com.apple.filesystems.msdosfs 1.10
com.apple.filesystems.smbfs 3.2.3
com.apple.driver.AppleTyMCEDriver 1.0.3d2
com.apple.driver.AGPM 110.23.37
com.apple.filesystems.autofs 3.0
com.apple.driver.AppleMikeyHIDDriver 131
com.apple.driver.AppleMikeyDriver 281.52
com.apple.driver.AppleUpstreamUserClient 3.6.5
com.apple.driver.AppleMCCSControl 1.5.5
com.apple.driver.AudioAUUC 1.70
com.apple.kext.AMDLegacyFramebuffer 1.6.8
com.apple.driver.AppleHDA 281.52
com.apple.AGDCPluginDisplayMetrics 3.20.13
com.apple.driver.AppleHV 1
com.apple.kext.AMDRadeonX3000 1.6.8
com.apple.driver.AppleLPC 3.1
com.apple.iokit.IOUserEthernet 1.0.1
com.apple.driver.ACPI_SMC_PlatformPlugin 1.0.0
com.apple.kext.AMD5000Controller 1.6.8
com.apple.iokit.IOBluetoothSerialManager 6.0.7f10
com.apple.driver.pmtelemetry 1
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.AppleOSXWatchdog 1
com.apple.driver.AppleIntelSlowAdaptiveClocking 4.0.0
com.apple.filesystems.apfs 748.51.0
com.apple.filesystems.hfs.kext 407.50.6
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.BootCache 40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0
com.apple.AppleSystemPolicy 1.0
com.apple.iokit.SCSITaskUserClient 404.30.2
com.apple.driver.AppleFWOHCI 5.5.9
com.apple.driver.AirPort.Brcm4331 800.21.30
com.apple.driver.Intel82574LEthernet 2.7.2
com.apple.driver.AppleAHCIPort 329.50.2
com.apple.driver.AppleRTC 2.0
com.apple.driver.AppleHPET 1.8
com.apple.driver.AppleACPIButtons 6.1
com.apple.driver.AppleSMBIOS 2.1
com.apple.driver.AppleACPIEC 6.1
com.apple.driver.AppleAPIC 1.7
com.apple.driver.AppleIntelCPUPowerManagementClient 220.50.1
com.apple.driver.AppleUSBLegacyHub 900.4.1
com.apple.nke.applicationfirewall 183
com.apple.security.TMSafetyNet 8
com.apple.security.quarantine 3
com.apple.driver.AppleIntelCPUPowerManagement 220.50.1
com.apple.driver.usb.cdc 5.0.0
com.apple.iokit.IOUSBHIDDriver 900.4.1
com.apple.iokit.IOUSBMassStorageClass 4.0.4
com.apple.driver.AppleUSBComposite 900.4.1
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 281.52
com.apple.kext.OSvKernDSPLib 526
com.apple.iokit.IONDRVSupport 519.20
com.apple.iokit.IOAVBFamily 680.2
com.apple.plugin.IOgPTPPlugin 680.15
com.apple.iokit.IOEthernetAVBController 1.1.0
com.apple.iokit.IOAcceleratorFamily2 378.26
com.apple.driver.IOPlatformPluginLegacy 1.0.0
com.apple.driver.IOPlatformPluginFamily 6.0.0d8
com.apple.driver.AppleSMBusController 1.0.18d1
com.apple.driver.AppleSSE 1.0
com.apple.iokit.IOSurface 211.15
com.apple.kext.AMDLegacySupport 1.6.8
com.apple.driver.AppleSMBusPCI 1.0.14d1
com.apple.iokit.IOFireWireIP 2.2.9
com.apple.driver.AppleHDAController 281.52
com.apple.iokit.IOHDAFamily 281.52
com.apple.iokit.IOAudioFamily 206.5
com.apple.vecLib.kext 1.2.0
com.apple.iokit.IOSerialFamily 11
com.apple.AppleGPUWrangler 3.20.13
com.apple.AppleGraphicsDeviceControl 3.20.13
com.apple.iokit.IOGraphicsFamily 519.20
com.apple.iokit.IOSlowAdaptiveClockingFamily 1.0.0
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 6.0.7f10
com.apple.iokit.IOBluetoothHostControllerUSBTransport 6.0.7f10
com.apple.iokit.IOBluetoothHostControllerTransport 6.0.7f10
com.apple.iokit.IOBluetoothFamily 6.0.7f10
com.apple.iokit.IOAHCIBlockStorage 301.40.2
com.apple.driver.usb.IOUSBHostHIDDevice 1.2
com.apple.driver.usb.networking 5.0.0
com.apple.driver.usb.AppleUSBHostCompositeDevice 1.2
com.apple.driver.usb.AppleUSBHub 1.2
com.apple.filesystems.hfs.encodings.kext 1
com.apple.iokit.IOSCSIMultimediaCommandsDevice 404.30.2
com.apple.iokit.IOBDStorageFamily 1.8
com.apple.iokit.IODVDStorageFamily 1.8
com.apple.iokit.IOCDStorageFamily 1.8
com.apple.iokit.IOAHCISerialATAPI 267.50.1
com.apple.iokit.IOFireWireFamily 4.7.2
com.apple.iokit.IO80211Family 1200.12.2
com.apple.driver.corecapture 1.0.4
com.apple.iokit.IOAHCIFamily 288
com.apple.driver.usb.AppleUSBEHCIPCI 1.2
com.apple.driver.usb.AppleUSBUHCIPCI 1.2
com.apple.driver.usb.AppleUSBUHCI 1.2
com.apple.driver.usb.AppleUSBEHCI 1.2
com.apple.driver.usb.AppleUSBHostPacketFilter 1.0
com.apple.driver.AppleUSBHostMergeProperties 1.2
com.apple.driver.AppleEFINVRAM 2.1
com.apple.driver.AppleEFIRuntime 2.1
com.apple.iokit.IOSMBusFamily 1.1
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.security.sandbox 300.0
com.apple.kext.AppleMatch 1.0.0d1
com.apple.driver.DiskImages 480.60.1
com.apple.driver.AppleFDEKeyStore 28.30
com.apple.driver.AppleEffaceableStorage 1.0
com.apple.driver.AppleKeyStore 2
com.apple.driver.AppleUSBTDM 439.70.3
com.apple.driver.AppleMobileFileIntegrity 1.0.5
com.apple.iokit.IOUSBMassStorageDriver 140.70.2
com.apple.iokit.IOSCSIBlockCommandsDevice 404.30.2
com.apple.iokit.IOSCSIArchitectureModelFamily 404.30.2
com.apple.iokit.IOStorageFamily 2.1
com.apple.driver.AppleCredentialManager 1.0
com.apple.driver.KernelRelayHost 1
com.apple.driver.AppleSEPManager 1.0.1
com.apple.driver.IOSlaveProcessor 1
com.apple.iokit.IOReportFamily 31
com.apple.iokit.IOTimeSyncFamily 680.15
com.apple.iokit.IONetworkingFamily 3.4
com.apple.iokit.IOUSBFamily 900.4.1
com.apple.iokit.IOUSBHostFamily 1.2
com.apple.driver.usb.AppleUSBCommon 1.0
com.apple.driver.AppleBusPowerController 1.0
com.apple.driver.AppleACPIPlatform 6.1
com.apple.driver.AppleSMC 3.1.9
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOACPIFamily 1.4
com.apple.kec.pthread 1
com.apple.kec.corecrypto 1.0
com.apple.kec.Libm 1
EOF
Model: MacPro5,1, BootROM MP51.0084.B00, 4 processors, Quad-Core Intel Xeon, 2,8 GHz, 6 GB, SMC 1.39f11
Graphics: ATI Radeon HD 5770, ATI Radeon HD 5770, PCIe
Memory Module: DIMM 1, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313132553741465038432D47372020
Memory Module: DIMM 2, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313132553741465038432D47372020
Memory Module: DIMM 3, 2 GB, DDR3 ECC, 1066 MHz, 0x80AD, 0x484D54313132553741465038432D47372020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x8E), Broadcom BCM43xx 1.0 (5.106.98.102.30)
Bluetooth: Version 6.0.7f10, 3 services, 27 devices, 1 incoming serial ports
Network Service: Ethernet 1, Ethernet, en0
PCI Card: ATI Radeon HD 5770, Display Controller, Slot-1
PCI Card: pci10ee,3fc6, Audio, Slot-2
PCI Card: DeckLink SDI 4K, Video, Slot-4
Serial ATA Device: HL-DT-ST DVD-RW GH41N
Serial ATA Device: WDC WD1001FALS-41Y6A0, 1 TB
Serial ATA Device: Patriot Burst, 480,1 GB
Serial ATA Device: SAMSUNG HD103SJ, 1 TB
Serial ATA Device: SAMSUNG HD103SJ, 1 TB
USB Device: USB Bus
USB Device: USB Bus
USB Device: USB Bus
USB Device: USB Bus
USB Device: USB Bus
USB Device: BRCM2046 Hub
USB Device: Bluetooth USB Host Controller
USB Device: USB Bus
USB Device: USB 2.0 Bus
USB Device: USB2.0 Hub
USB Device: USB 2.0 Bus
USB Device: KVM-via-IP Extender
USB Device: USB Receiver
USB Device: USB Multimedia Keyboard
USB Device: USB-PS/2 Optical Mouse
USB Device: Virtual Keyboard
FireWire Device: built-in_hub, Up to 800 Mb/sec
Thunderbolt Bus:
I'm not sure this server message is related, there are other active clients:
[616711.563320] usb 2-1.2.3.4: usbfs: process 29533 (vhusbdx86_64) did not claim interface 0 before use
[616711.563903] usb 2-1.2.3.4: usbfs: interface 0 claimed by usbfs while 'vhusbdx86_64' sets config #1
[618128.907845] usb 2-1.4.3.3: usbfs: process 7168 (vhusbdx86_64) did not claim interface 0 before use
[618128.908259] usb 2-1.4.3.3: usbfs: interface 0 claimed by usbfs while 'vhusbdx86_64' sets config #1
.
Thanks for the logs, the virtualhere driver version 2.0.3 is the correct latest version. It appears to be crashing in another driver, a USB cdc driver as that was the last loaded kernel extension and virtualhere doesnt appear in the kernel panic stack trace at all. I havent heard of any other reports of virtualhere crashing with iloks.
I dont think its related to virtualhere, because the crashing driver appears to be a cdc driver, that is used for USB -> Serial devices.
Do you have one of those plugged into your mac usb port directly at the client? If so that might be the culprit.
Is there a reason why you upgraded to 10.13 instead of e.g 10.14 (or even 10.15 which came out yesterday). Ideally i think you should try to upgrade again to 10.14 if you can. 10.15 might be a bit risky at this point as osx is quite buggy for brand new versions. But 10.14 will be better than 10.13 regarding usb
The MacPros would need a new
The MacPros would need a new GPU (which in turn won't display the UEFI boot screen) to work with 10.14 and there are application compatability concerns, so I'd prefer to stick with 10.13.
I'll try to reproduce the panic, it has been happening on 3-4 machines.
Can you make anything of the application crash (syslog section above)?
.
Does virtualhere always crash sometime before the panic?
I didn't get any panics today
I didn't get any panics today but three crashes with this message (after closing the GUI error message):
USBF: 8074.849 WARNING, com_virtualhere_vhhcduc::clientDied
USBF: 8074.849 WARNING, com_virtualhere_vhhcd::userspaceCrashed (unbinding all devices)
USBF: 8074.855 com_virtualhere_vhhcd::hubResetChangeConnection port[1] which was functionAddress 2 and remoteAddress 4, has disconnected
USBF: 8074.855 com_virtualhere_vhhcd::hubResetChangeConnection port[2] which was functionAddress 3 and remoteAddress 5, has disconnected
USBF: 8074.856 com_virtualhere_vhhcd::hubResetChangeConnection port[3] which was functionAddress 4 and remoteAddress 6, has disconnected
.
When i get back from holiday next week, ill send you a debug build of the client.
.
Could you try this debug build https://www.virtualhere.com/sites/default/files/usbclient/test/VirtualH…
Exit the current client first then run this version. If/when it crashes, send me that part of the syslog with the backtrace, either post it here or email me at mail [at] virtualhere.com (mail[at]virtualhere[dot]com) Thanks!