VMWare and the latest Linux kernels

Last week I pulled the 4.0.0-2 kernel from the Debian testing repositories and started using it on my main box. Yesterday morning I try to start VMWare Workstation 11 for the first time since upgrading the kernel, only to have the driver install fail. Some poking around revealed that the network driver was failing because of apparent changes in the kernel headers.

I manually unpacked the driver source (located in /usr/lib/vmware/modules/source) and started debugging the compiler errors. I found four:

  1. One of the VMWare headers defined PCI_VENDOR_ID_VMWARE, which is in the kernel headers as of 4.0.
  2. The function alloc_netdev gained an additional argument in 3.17.
  3. The driver code was referring to the f_dentry member of struct file. This member was a #define that was removed in 3.19.
  4. The driver code was referencing function skb_copy_datagram_iovec, which was also removed in 3.19.

The first three were easy enough to fix. For the first, just remove the redundant #define from the driver header. For the second, add NET_NAME_UNKNOWN as the third argument to alloc_netdev (I got that from StackOverflow, I think). For the third, replace the reference to f_dentry with the code that used to be in the #define.

The fourth was trickier. There was no clear replacement for the removed function. Fortunately, I was able to track down a recent post on the VMWare message boards asking about these same issues. That gave me the solution for the third problem: replace the missing function with the following:

struct iov_iter to;
iov_iter_init(&to, READ, &iov, 1, len);
return skb_copy_datagram_iter(skb, 0, &to, len);

To resume the driver install after making these changes, I repackaged the driver source into the tarball in the VMWare source directory (command tar -cvf vmnet.tar vmnet-only/) so the VMWare driver installation program would use it and finish installing the other components. I had no problems with anything else in the driver builds, and was able to start my Windows VM with no issues using the modified source.

I should note that one of the commenters on the VMWare board mentioned that the proper fix for the filep->f_dentry line is to replace

if (filp && filp->f_dentry) {
    inode = filp->f_dentry->d_inode;
}

with

inode = file_inode(filp);

I did not do this, since I just wanted a quick-and-dirty fix for the driver so I could get to my VMs again.

links

social