Sniffing the Fuji G mount with a MCEX-45G macro tube

2022-09-19T00:00:00+00:00

Context</a></h1>
The Fujifilm GFX line of cameras is the most accessible digital 'medium format' system to date, and the Fujinon lenses are optically fantastic and incredibly capable.

While there are a handful of dumb mechanical lens adaptors for the G-mount, and a couple of 'smart' adaptors from companies like Fringer, there is very little information about the mount or protocol.
I had a 'the right kind of wrong' idea to figure out GF lens control with the end-goal of (possibly) building a 'medium format' film camera around them.
As all of the currently released GF lenses use fly-by-wire focus, electronic aperture, and some need to be powered up to position floating optical elements - if I want to build a camera around them then I'll need to reverse engineer the mechanical, electrical, and software to even consider camera design.
Disassembling a MCEX-45G</a></h1>
I picked up the 45mm extension tube on a ~25% discount with the intention of using it to bootstrap debugging, and probably provide donor lens/body mount flanges.
Can confirm it works as advertised with the GF 110mm F2.

Lets dive in.
Lens mount basics</a></h2>
The electrical connections between the body and lens are made with 12 spring loaded contacts on the body which mechanically align with 11 contact rectangles on the lens.
My reference drawing for the lens contacts:

Lens Side</a></h3>

The lens-side contacts are 1.6 x 1.3 mm rectangular pads, with the long edge aligned tangentially to the circular mount. </li>

When looking at the back of the lens mount, the right-most pin is a 1.3 mm x 1.3 mm square contact.

For ease of convention, I'll label this square pad as `Pin 1</code>, though I don't know what the official nomenclature is.</li> </ul> </li>`
`Based on this numbering scheme, Pin 5</code> and Pin 6</code> are mechanically the same contact rectangle, and therefore electrically common.`
`Unlike the body side mount, the lens mount/bayonet is not electrically common with Pin 5</code> and Pin 6</code>.</li> </ul> </li>`One interesting point to note - the pinout for the GF 45mm (top) doesn't use all 12 contacts that the body provides, while the GF 110mm (bottom) does. </li> </ul> Body Side</a></h3> The spring section of the pins have a 1.0 mm diameter, and a maximum achievable travel of 0.8 mm.</li> The spring pins have a conical/round tip.</li> The pins are spaced 6° apart, for a total span/range of 66°. There is no pin on the vertical axis, and from a vertical datum aligned with the center of the lens, the outermost edge pins are aligned to +33° and -33°.</li> The mount (metal ring/bayonet assembly) is electrically common with the exposed metal inside the battery compartment, as well as Pin 5</code> and Pin 6</code>.</li> </ul> Now lets start opening up the extension tube to learn more... Tube design</a></h2> From the rear, a few screws remove the bayonet mount (i.e. the 'lens side mount'). The weathersealing gasket slides off with it, and we can see the simple internal design of the extention tube. Looking into the cavity, there are some small screws which allow the removal of the internal baffle cylinder piece. The 'female' flange assembly which is found on the camera body has a machined mating face and orange lens alignment dot, backed by a thin spring plate to provide preload against the incoming lens bayonet features. The lens release is a simple spring loaded post that just rests against the main tube assembly. The main tube assembly is actually two pieces - the thin exterior 'skin', and a thicker machined aluminium piece which has most of the mounting geometry for the mounting flanges and electrical sub-assembly. CAD Models</a></h2> I took a handful of measurements of each mount and attempted to draw mechanically compatible lens and body mount designs in Solidworks. I'm not 100% on the accuracy of either design, but a motion study showed them mating successfully and I'm confident I'd be able to use these as a starting point for a more manufacture-friendly design. Flex PCB</a></h2> As we expected before the tear-down, the extension tube only provides a dumb pass-through to the lens mount, there's no active electronics. The pad sections of the flex are held captive against a plastic a ring which supports some tiny pogo-pins. Hindsight tells us that unscrewing the plastic retention bracket behind the pogo-pins is a mistake - I spent the better part of an hour trying to guide the springs back into place with a handmade jig of needles. We now have some good hints of which pins are responsible for power, and those which are intended for data. Attaching debug probes</a></h2> With the electrical sub-assemblies out of the tube we begin by removing the Kapton tape protecting the through-hole pins, solder silicone wires onto the camera side of the flex, and then re-apply fresh Kapton tape to protect the joins from electrical contact with the tube/mount. But you might be asking, "how do you get the wires out of a sealed tube"? With the exit holes measured and drilled, I started re-assembling the extension tube. Once the tube was re-assembled, I identified each debug wire by checking for continuity with the lens mount pins, and then soldered on a terminal strip to make debug connections easier. Electrical Behaviour</a></h1> We'll start by identifying our power pins. Some immediate clues can be gleaned from looking at the flex PCB and by probing the pins on the camera. The ground was assumed to be Pin 5</code> as it has higher-current traces and is common with Pin 6 on the lens mount side. It's also electrically common with the GFX body mount. After poking around with a multimeter and oscilloscope for a few minutes, we have a rough pinout: Pin</th> Voltage</th> State</th> Scope/Notes</th></tr></thead> 1</td> ~0V</td> </td> Never triggers scope, even with varied stimulus/settings.</td></tr> 2</td> 5.32V</td> DC</td> </td></tr> 3</td> 6.72V</td> DC</td> </td></tr> 4</td> 8.01V</td> DC</td> Battery Voltage. ~100mV ripple when focus motor engages.</td></tr> 5</td> -</td> -</td> Used as ground connection.</td></tr> 6</td> -</td> -</td> Electrically common with Pin 5</td></tr> 7</td> 3.4V</td> Normally high</td> Infrequent squarewave. No recognisable pattern.</td></tr> 8</td> 3.38V</td> Normally high</td> High speed edges, infrequently triggered. Pulses LOW for approx 100-350ns at a 10ms interval.</td></tr> 9</td> 0V</td> Normally low</td> Squarewave, data line.</td></tr> 10</td> 3.3V</td> Normally high</td> Squarewave, clock line at 1.5MHz.</td></tr> 11</td> 0V</td> Normally low</td> Squarewave, data line.</td></tr> 12</td> 3.3V</td> Normally high</td> Squarewave, goes low when iris closes.</td></tr> </tbody></table> I now know what's safe to connect to the logic analyser and could start sniffing for communications between the body and lens. Signal analysis</a></h2> A capture of some simple camera operation will start off our analysis: Power up,</li> Focus, then focus+photo</li> Power off</li> </ul> Data Lines</a></h2> Lets start by looking into the data lines first. We can see some one-way data transfers, as well as request/response style behaviour. This lets us make the following observations about these data pins: Pin 9</code> is the body's DATA OUT line,</li> Pin 10</code> is a 1.5MHz SPI-style clock signal,</li> Pin 11</code> is the body's DATA IN line,</li> </ul> With this in mind, lets identify the properties of the communications bus and determine if it's a fairly typical SPI or some Fuji special sauce: A transaction seems to consist of 32-bits at minimum, Almost all transactions are 32-bits long.</li> During startup, a pair of 1048-bit long transactions occur with a 3ms gap - these are the longest identified transactions.</li> As the division of 1048</code> and the potential bits-per-transfer should be an integer, we can reasonably throw out 16</code> and 32</code> bit transfer sizes.</li> Therefore it's reasonable to assume byte-level layout of packets.</li> </ul> </li> We can also determine the clock behaviour, by looking at the first falling edge and any 8-bit aligned end: The clock is high when inactive (CPOL = 1).</li> Data appears to be timed for the clock's trailing edge (CPHA = 1)</li> </ul> </li> </ul> Other IO</a></h2> Pin 1</code> is intended as a signalling line due to the trace thickness, but I saw no activity. Pin 7</code> seems to be related to auto-focus motor activation. Pin 12</code> correlates to the iris closing. Protocol Analysis</a></h1> We can now sniff the packets being sent between the lens and body. From a few test captures, there's a few key packet sequences that are immediately obvious. Startup/Identification packets</a></h2> After the camera is powered on with a GF 45mm F2.8 lens, a series of 131B</code> transactions occur (each sent twice 3ms apart). These transactions are from the lens to the body and describe the lens. The first transaction starts with 0x00 0x00</code>, the second transaction starts with 0xFF 0xFF</code>. Both end with what I'm assuming is a single checksum byte. The payload (hex formatted) looks like this: 4c52 3130 3641 0000 4653 534e 5730 3036 4746 3435 6d6d 4632 2e38 2052 2057 5200 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 3234 3033 3338 3636 3035 0156 0100 0100 0100 0100 c801 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 </code></pre> Doing a little bit of conversion we find: The first chunk of data is ASCII: LR106A FSSNW006GF45mmF2.8 R WR</code> Not sure what the first string section means yet.</li> Part of the last section is obvious as the lens is officially labelled as GF45mmF2.8 R WR</code>.</li> </ul> </li> The second block of data in the payload is likely some combination of binary data and potentially ASCII formatted numbers with no obvious meaning.</li> </ul> Sniffing the GF 110mm F2 startup packet we see similar behaviour: 4c52 3130 3441 0000 4653 534e 5731 3034 4746 3131 306d 6d46 3220 5220 4c4d 2057 5200 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 3035 4330 3030 3234 0000 0160 0110 0110 0110 0110 c801 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 </code></pre> The first section of data is ASCII: LR104A FSSNW104GF110mmF2 R LM WR</code></li> The second block is different and similarly unrecognisable.</li> </ul> After the lens packets, and only when the GF110mm is mounted, the camera body describes itself to the lens. 5350 5833 0000 0000 0000 4746 5820 3530 5200 0000 0000 0000 0000 0000 0000 4746 5820 3530 5200 0000 0000 0000 0035 3933 3533 3433 3833 3633 3131 3831 3131 3339 3744 3031 3031 3130 3834 3102 2001 3501 0001 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 </code></pre> The first section of data is ASCII: SPX3 GFX 50R GFX 50R</code> (whitespace trimmed) Unsure of what SPX3</code> represents.</li> The repeated pair of GFX 50R</code> string sequences match the body under test.</li> </ul> </li> The second chunk of data is another unknown mix of potentially ASCII, and unprintable bytes.</li> </ul> 'Idle' packets</a></h2> When the camera is powered on but is otherwise sitting idle, we see a burst of transactions occur every 40ms. These packets represent the bulk of 'noise' when reading packet traces, so understanding, decoding and then filtering them represents a worthwhile effort. Transmission #</th> Camera</th> Lens</th></tr></thead> 1</td> 0x00 0x00 0x08 0x20</code></td> -</td></tr> 2</td> 0x?? 0x10 0x80 0x??</code></td> 0x08 0x00 0x88 0x32</code></td></tr> 3</td> -</td> 0x03 0x80 0x08 0x3C</code></td></tr> 4</td> 0x08 0x00 0x88 0x??</code></td> 0x?? 0x00 0x80 0x??</code></td></tr> </tbody></table> On every third burst (120ms interval), we have 2 additional transactions for a total of 6 total. Transmission #</th> Camera</th> Lens</th></tr></thead> 1</td> 0x00 0x00 0x08 0x20</code></td> -</td></tr> 2</td> 0x?? 0x10 0x80 0x??</code></td> 0x08 0x00 0x88 0x32</code></td></tr> 3</td> 0x00 0x00 0x09 0xA6</code></td> 0x03 0x80 0x08 0x3C</code></td></tr> 4</td> 0x?? 0x00 0x88 0x??</code></td> 0x?? 0x00 0x89 0x??</code></td></tr> 5</td> -</td> 0x00 0x15 0x09 0x9A</code></td></tr> 6</td> 0x08 0x00 0x89 0xB8</code></td> 0x?? 0x00 0x80 0x??</code></td></tr> </tbody></table> For both the 4 and 6 packet bursts, when I marked the first bytes as 0x??</code> this was to represent them varying between subsequent bursts. Sampling 5 random sequences of packets gives the impression that there isn't obvious packet counting behaviour though. The final bytes marked 0x??</code> are an assumed CRC byte which is changing due to the cycling first byte... Iris packets</a></h2> Looking for packets we don't recognise around the IRIS control IO line (Pin 12</code>) changing state, I found the following series of transactions. This whole sequence spans 275 ms. Transmission #</th> Camera</th> Lens</th></tr></thead> 1</td> 0x12 0x40 0x18 0xBA</code></td> -</td></tr> 2</td> 0x09 0x10 0x80 0x2A</code></td> 0x08 0x00 0x98 0xB6</code></td></tr> 3</td> 0x00 0x00 0x3F 0xC6</code></td> 0x12 0x40 0x18 0xBA</code></td></tr> 4</td> 0x0A 0x00 0x98 0x86</code></td> 0x09 0x00 0xBF 0xE0</code></td></tr> Pin 12</code> (IRIS) low</td> </td> </td></tr> 5</td> 0x00 0x00 0x00 0x00</code></td> 0x00 0x00 0x3F 0xC6</code></td></tr> 6</td> 0x08 0x00 0xBF 0xD8</code></td> 0x0A 0x00 0x80 0x22</code></td></tr> Silent for 40ms</td> </td> </td></tr> </td> 6 bursts of IDLE packets</td> </td></tr> Pin 12</code> (IRIS) high</td> </td> </td></tr> </tbody></table> These packets add some confidence to a gut feel that the first byte is acting as some kind of identifier - we can see the camera send packets to the lens and receive packets in matching order, however the middle two bytes vary. To get any deeper meaning out of this I think I'm going to make/find some better tools for packet analysis and experimentation. What's next?</a></h1> 2024 Update: I've made the GitHub repo public</a> as a few people have asked questions. Related mechanical post continues the process</a> </blockquote> There are three key areas which need to progress as part of the reverse engineering effort: 3D print or machine some test lens mount assemblies to check compatibility,</del></li> Better understand the behaviour of the extra IO signals, and measure current consumption on the various DC supply lines in anticipation of making functional interface electronics,</li> Develop a SPI man-in-the-middle bridge to build out tooling and firmware as part of reverse-engineering the communications protocol. Being able to detect, log and then filter out common packets should make finding unique packets easier, and give clues as to what might be inside.</li> </ul> </li> </ul> Building an offsite Proxmox Backup Server 2022-06-10T00:00:00+00:00 Why an offsite PBS</a></h1> VM backups can be done to external SMB/CIFS drives directly from Proxmox. I was using an older QNAP NAS for backups but had a few workflow issues I wanted to resolve, namely the full-copy for each backup run presenting two issues: a month's backups would consume ~3TB, and</li> waking up to hear the backup still running after it's automated midnight start</li> </ul> Proxmox Backup Server</a> (PBS) is a companion service which provides offsite de-duplicated backups for Proxmox VE container and virtual machine images. The main benefit of running a PBS instance over adding some networked storage to Proxmox is integration and control: de-duplicated backups reduce raw disk space over longer timeframes especially pronounced for large VM images containing static content</li> </ul> </li> Chunking means that only deltas are sent,</li> Compression on each end makes internet-based backups viable with Australian internet connections</li> </ul> Also, better integration with Proxmox's UI makes managing backups and restoration easier. With more than two dozen LXC and VM instances running on my main VM machine, I was wanting to get something running soon. Hardware</a></h1> I picked up 4 WD 14TB external drives for a reasonable price - not /r/datahorder bargains, but pretty affordable for Australia's e-retailers. For a small, reasonably quiet and turnkey server, I picked up a HP Microserver Gen10+ with an Intel Xeon E-2224 and 16GB of RAM. I used a PCIe to M.2 card for internal storage, and loaded the drives. Overall, pretty happy with the compact footprint and build quality of the G10+, though I'd love an internal M.2 slot... Overcomplicating it</a></h1> The destination for this backup server is offsite (a friend's house) - so there are some considerations for remote access that force some key software choices, as well as a desire to allow 'multi-tennant' operation where simple VM hosting and storage is allocated for use as a NAS. Additionally, I wanted to make the unit as close to set and forget as possible - no specific network setup, port-forwarding required. Therefore, Solid ZFS support is a must,</li> Some kind of virtualisation is needed for multi-user containers</li> A site-to-site VPN will be needed to allow the Proxmox VE server and PBS instances to communicate,</li> Networks and services should be sufficiently isolated that no access to either 'home' network can leak into any other,</li> I need the ability to access Proxmox's management plane remotely,</li> and what about remote access to HP's iLO management interface?</li> </ul> I tried TrueNAS at first, motivated by the ability to run the OS from a USB on the internal USB port, and wanting to see what everyone's raving about. While it's fine, it didn't feel like the correct solution for me, but I can see the appeal of setting ZFS and network shares from a GUI. Proxmox Setup</a></h1> Sticking with the tried and true Proxmox for a hypervisor was an easy choice - adding multiple users with pools of containers/VMs solves the multi-tenant requirements reasonably well, and ZFS volumes can be shared to the various containers with bind-mounts. ZFS Storage</a></h2> ZFS raidz1 will span the 4*14TB disks, supporting a single disk failure without loss of data, along with the benefits of bitrot detection, compression and CoW benefits... raidz1 is thought of as a"diagonal parity RAID", where parity blocks are distributed across all disks, with a total quantity of parity equal to one disk. </blockquote> The main storage zpool is called tank</code>.</li> datasets on tank</code> provide segregated storage called user1</code>, pbs</code> and user2</code>.</li> The internal nvme boot disk also uses single-disk ZFS. Called rpool</code> on the host.</li> </ul> Identify the disks, # ls /dev/disk/by-id ata-WDC_WD140EDGZ-11_REDACTED_5UG ata-WDC_WD140EDGZ-11_REDACTED_3WG ata-WDC_WD140EDGZ-11_REDACTED_L2N ata-WDC_WD140EDGZ-11_REDACTED_DAC </code></pre> Create the zfs pool with those disk ID's, and do some minimal config zpool create -f tank -o ashift=12 raidz1 ata-WDC_x_5UG ata-WDC_x_3WG ata-WDC_x_L2N ata-WDC_..._DAC zfs set compression=lz4 tank zfs set atime=off tank zfs set acltype=posixacl tank zfs set xattr=sa tank </code></pre> The idea is to create nested zfs file system datasets for each type or group of data. This helps with isolation, tuning, backups etc. zfs create tank/user1 zfs create tank/user2 </code></pre> These should be visible inside the default mount points on root, i.e. ls /tank/user1</code> Additional datasets can be nested using the same path notation. ZFS allows datasets to have different compression/encryption/block size parameters, and I use the deepest dataset level as the 'thing I'd pass to into a container'. Sharing storage via bind-mount</a></h2> To make filesharing half sane with various subsystems, create a series of groups on the host. These groups then have users assigned. These users are also created in the relevant end-user VM or LXC with the same GID:UID properties, allowing for sane permissions management and mapping. This adds a level of control over the downstream user's access to various sections of the storage solution. i.e. a 'fileserver' LXC user responsible for providing network access via SAMBA/NFS shares the user:group permissions of directories/shares which make sense to expose as shares, whereas a gameserver container would only be provided access and permissions to a specific dataset or subfolder etc. </blockquote> We manually specify the group ID's on the host to make things a bit easier to track. These are offset by 10000</code> to account for LXC unprivileged containers. On priv containers those groups can be created as needed. Use getent group</code> to list the groups and their ID's. addgroup --gid 101020 storage-user1 addgroup --gid 101030 storage-user2 </code></pre> Use groupdel group_name</code> to delete a group. To apply the group over a directory, set permissions, and set the access control list: chgrp -R storage-user1 /tank/user1/ chmod -R 2775 /tank/user1 # Optional if supported (not on the datasets I created this time) setfacl -Rm g:storage-user1:rwx,d:g:storage-user1:rwx /tank/user1 chgrp -R storage-user2 /tank/user2/ chmod -R 2775 /tank/user2 </code></pre> When access to storage is needed in a LXC, bind mount setup is pretty straightforward. From the host's console, navigate to /etc/pve/lxc</code> and edit the 101.conf</code> or file matching the LXC we just made. Add the mount points as required as a new line each: mp0: /tank/user1,mp=/mnt/user1,size=0 </code></pre> Start the container up, and navigating to /mnt</code> should show the bound folder folder. We then need to map the UID and GID between the host and the container to allow consistent file permissions on the mapped directories. On the LXC, addgroup --gid 1020 storage-user1 usermod -aG storage-user1 root </code></pre> Log-out and back in so the permissions can take effect. Test access by navigating to /mnt/user1</code> and create/edit some simple files. NAS container</a></h2> Providing access to the file system for non-PBS backups is a partial use-case, so Samba and rsync are needed. I setup a tiny Ubuntu LXC to handle these roles. Red Hat's Cockpit</a> is a lightweight, reasonably unobtrusive way to monitor and administer linux systems with a web GUI. It's strength is the design philosophy of using standard Linux services/config files, and only runs while a socket is made to the server. Ubuntu instructions are here</a>, but as a quick reference: apt install cockpit curl </code></pre> It will run on port 9090</code> and uses the container's Linux account credentials system for login. 45drives maintain some cockpit plugins to make identity and samba management easier. 45drives/cockpit-identities</a> and 45Drives/cockpit-file-sharing</a> need to be installed. curl -sSL https://repo.45drives.com/setup | sudo bash apt update apt install -y cockpit-identities cockpit-file-sharing cockpit-navigator </code></pre> For optional Windows/network discover-ability, install wsdd</a> from Ubuntu's package manager directly. apt install wsdd </code></pre> Samba share setup is reasonably standard at this point (i.e. frustrating to get perfect). Network Setup</a></h1> Isolation by design</a></h2> At home, I segregate end-user devices from servers and management interfaces with VLANs, but I was keen to avoid a similar approach on the remote site to separate the 'backup DMZ' from the local network. The plan is reasonably intuitive: Assign a second network adaptor to act as my 'backup DMZ'</li> Setup a site-to-site VPN with Wireguard to connect the remote and home networks.</li> Allow traffic flow across the Wireguard bridge,</li> Setup a static route on my local network to allow me to access the remote DMZ network as if it were local.</li> </ul> Because the Microserver has 4 gigabit ethernet ports in addition to the management interface, I can also connect directly to the 'isolated' network if I have hardware access. This would also make a lot of sense in a multi-wan environment. I'm aware this might be considered sub-optimal from a best-practices standpoint, but I'm happy enough with the level of security for a homelab use-case. Talescale is an alternative option that can provide machine-to-machine connections without the same network bridging. </blockquote> Site-to-site VPN</a></h2> To remotely access the backup server and it's network, I run a Wireguard container which automatically attempts to connect to the Wireguard instance running on my VM host at home. Because the connection is outbound only (aka a road-warrior), I don't need to configure port-forwarding on the remote network, and it's a controlled single point of contact between my 'backup DMZ' and the local network it's squatting in. In Proxmox's network page, I created an additional Linux bridge which was attached to the second ethernet adaptor, and assigned it 192.168.20.1/24</code>. This vmbr1</code> interface is intended to act as the only network interface to my containers. Due to the 'no remote network configuration' requirement, I did need to provide both interfaces to the Wireguard container for internet access. By adding the adaptor in this manner, Proxmox makes it's management interface available at 192.168.20.1</code>, which is useful if connecting via hardware on eno2</code>, and will be useful later when we want to access Proxmox over our bridge. First, we'll add another client to the existing onsite Wireguard container to allow remote access, ### begin hpems-offsite ### [Peer] PublicKey = Gredacted= PresharedKey = EredactedA= AllowedIPs = 10.6.0.8/32, 192.168.20.1/24 PersistentKeepalive = 25 ### end hpems-offsite ### </code></pre> Then start preparing the offsite container. Offsite setup</a></h3> A simple Debian LXC container was spun up, and both Proxmox virtual bridges were added to it's network configuration. Once wireguard was installed, the /etc/wireguard/wg0.conf</code> configuration looks something like this: [Interface] PrivateKey = Gredacted2= Address = 10.6.0.8/24 MTU = 1200 DNS = 192.168.1.1, 9.9.9.9 PostUp = iptables -A FORWARD -i wg0 -j ACCEPT PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostUp = iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE # Push two packets across the tunnel after opening it PostUp = ping -c2 192.168.1.154 PostDown = iptables -D FORWARD -i wg0 -j ACCEPT PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -t nat -D POSTROUTING -o eth1 -j MASQUERADE [Peer] PublicKey = EredactedK= PresharedKey = FredactedA= Endpoint = redacted.duckdns.org:51820 AllowedIPs = 10.6.0.8/24, 0.0.0.0/0 PersistentKeepAlive=15 </code></pre> The interface and peer are reasonably normal, but we've got a few extra bits of iptables</code> network configuration in the PostUp</code> and section which allow NAT traversal between the eth0</code> and eth1</code> interfaces. Additionally, once the link is up, we ping the onsite VPN container to ensure some traffic is flowing over the VPN. The PostDown</code> sections provide cleanup of the PostUp</code> changes. We also need to edit /etc/sysctl.conf</code> and uncomment net.ipv4.ip_forward=1</code> to allow IPv4 forwarding across the container's networks. Activate the change with sysctl -p</code>. Enable the VPN with wg-quick up wg0</code> and wg-quick down wg0</code> as typical. Also remember to open the wireguard link at startup. sudo systemctl enable wg-quick@wg0.service sudo systemctl daemon-reload </code></pre> I originally had wg-quick</code> fail with /usr/bin/wg-quick: line 32: resolvconf: command not found</code>. This was resolved by providing systemd's resolver. I did this via symlink, ln -s /usr/bin/resolvectl /usr/local/bin/resolvconf </code></pre> </blockquote> Onsite Setup</a></h3> We'll also need to make sure traffic can be forwarded across the VPN, # Masquerade rules PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE </code></pre> On my home network, I setup a static route on my Unifi USG router to allow traffic flow between my local Wireguard container's IP and the subnet used on the bridge network backing the remote DMZ (192.168.20.0/24</code>) This means that traffic attempting to access IP addresses in the remote DMZ subnet will be routed to the onsite Wireguard container. We can see this redirection in action by running a traceroute between my workstation on the local network and a container on the remote DMZ, scott@octagonal ~ $ traceroute 192.168.20.2 traceroute to 192.168.20.2 (192.168.20.2), 64 hops max 1 192.168.1.1 0.410ms 0.327ms 0.331ms 2 192.168.1.154 0.843ms 0.403ms 0.392ms 3 192.168.20.2 9.962ms 8.651ms 9.945ms </code></pre> We can also access the remote Proxmox management interface by browsing to https://192.168.20.1:8006</code>. Accessing the internet in remote DMZ containers</a></h3> Because the remote machine has a separate but bridged network for all of my services, they don't have direct access to the local internet (by design). Additionally, because there's no software router running on the 192.168.20.1/24</code> subnet, the remote containers need to be told how to access the 'onsite' 192.168.1.1</code> resources via the bridge available at 192.168.20.2</code>. For ubuntu: ip route add 192.168.1.0/24 via 192.168.20.2 dev eth0 </code></pre> For debian: ip route add 192.168.1.0/24 via 192.168.20.2 dev eth0 onlink </code></pre> This does have the minor downside that internet traffic on these remote containers is taking a pretty indirect route to the internet, but it's only really for software updates. PBS</a></h1> The setup of PBS in an LXC</a> was fast and easy, so I won't go into too much depth, Provisioned a Debian container with 2 cores and 256MB of RAM</li> Installed PBS installed via the official channels.</li> Setup a bind-mount for a ZFS volume, and configured a datastore at the path in /mnt/pbs</code></li> Get the PBS fingerprint from the dashboard, and use it to add a new storage volume to the Proxmox VE machine pointing at the container's IP.</li> Configure backup frequency and rules,</li> On the PBS webpage, setup prune and verification jobs to run periodically on the datastore.</li> </ul> The first backup ran overnight as the HP microserver only has a gigabit link, but since then I've been running remote backups every Monday, Wednesday and Saturday starting at 12:30am. With my 50Mbps upload speed, the incremental backups normally take an hour or two to send the ~40GB deltas. The PBS interface is reasonably well laid out, allowing visibility of the various containers/VM's and the different snapshots. One neat benefit is the ability to introspect a LXC container's file system from the web interface, which might be useful to cherrypick a specific file instead of running a full restoration of that container. Closing thoughts</a></h1> Whilst not the cheapest or most practical for all home-labs, I've got a far greater level of comfort by both improving the backup experience for Proxmox, as well as adding offsite file storage. The setup of a full-time bridged remote network was a first for me, and I can't say enough good things about the stability of the system and functionality available with modern open-source options. Update: More than a year later everything is still going strong. I wouldn't have changed my approach, other than setting up ZFS send/recv for backing up my other files with snapshot support, instead of relying on rsync. I've restored a few containers without issue, and average a de-duplication factor of about 12. </blockquote> Tearing down a dead NEMA-23 Clearpath SC 2021-12-22T00:00:00+00:00 But how???</a></h1> Cause of death - 6+ months of infrequent exposure to 'fresh' water. This particular model did have the additional waterproofing shaft seal, but the electrical connectors weren't sealed with much care. When it started throwing tracking error faults randomly, it was time to replace it. Looking inside a wet Clearpath</a></h1> I've used a handful of different sized Teknic Clearpath servos from the MC, SD and SC ranges, and an opportunity to take a peek inside the integrated drive mounted to the back of the servo was too hard to resist. Undoing the screws lets us pull the integrated drive unit off the back of the motor and frame. Mounted to the servo side of the assembly is a PCB which acts as a mounting interface for the optical encoder and interfaces to the motor's windings. The rest of the IO and drive electronics sit in the cast heatsink. We can already see some corrosion products sitting near the 'bottom' of the heatsink. Removing 6 more hex-screws separates the electronics stack. A better look at the extent of the corrosion products... The 'front' board has the optical encoder, IO and some power conditioning. The back of said board has mostly IO conditioning optocouplers etc. The control microcontroller, gate drivers/current shunts/FETs are on the rear-most PCB. The rear-face of the control PCB has a cute thermal interface from the rest of the FETs, to a nicely machined step on the cast heatsink. The USB config port is also mounted here, along with some bulk capacitors which have pockets in the heatsink. Closing thoughts</a></h1> Running this servo so far out of spec in a wet environment worked far better than it had any right to, and packing the IO connectors of the replacement servo with dielectric grease seems to have prevented any repeated damage. For the price, I've found the form factor and reliability of the Clearpath servos to make them a good choice for precision positioning, provided the mechanical system has sufficient external gearing and low total backlash. The internal electronics design and integration reflects the solid design and build quality. Just wish the config and tuning software ran on something other than Windows...

26oclock

Sniffing the Fuji G mount with a MCEX-45G macro tube

Building an offsite Proxmox Backup Server

Tearing down a dead NEMA-23 Clearpath SC