“Upgrading” To 10 Gigabit Networking Was Challenging

I decided to ‘upgrade’ to 10 Gigabit home networking as part of upgrading my content creation workflow series, but as it turns out, 10Gb can be a difficult upgrade. Achieving 1 Gigabit is a LOT easier.

Soin this videoI discuss and show all the difficulties I faced when trying to move to 10 Gigabit networking, starting with PCIe and mATX/ATX PC builds but also discussing 10 Gig switches and cabling. I also run through some video editing tests and benchmarks that show exactly how much bandwidth I actually use when video editing over the network.

If you prefer text over video, please read on for the guide/transcript version of this video.

Video Transcript

Hey everyone, THIS is a 10 Gigabit networking switch (well actually it’s an empty box), but after setting up the ACTUAL switch over there, I was able to save an MASSIVE 20 seconds of time, each week when video editing. Wait, what? Okay so as it turns out, upgrading to 10 Gig networking is HARD and expensive (if you do it right) – and in SOME cases it might be worth sticking to just 1 or 2 and a half Gig networking, so here’s some of the difficulties I came across – and I’ll also discuss whether I think it was worth paying the money and upgrading in the end.

Not All PCIe Cards Are The Same

So firstly, when I “knew” that I simply HAD to upgrade to 10 Gigabit networking… because I developed FOMO from reading the/r/Homelab subreddit… I went looking at 10 Gigabit PCIe cards, and I found that they were REALLY expensive. So I went on to eBay and I purchased two usedX540-T2 networking cardsfor £30 each. While these WERE a good enough deal, there’s probably a few reasons why the original owners actually sold them. Firstly they are fairly old cards running PCIe gen 2 – which might not SOUND too bad (because if they work, they work, right?). BUT since gen 2 has lower bandwidth across the board, you need an x4 slot for them – otherwise you’ll only get 1 Gigabyte or less speeds, which will be capping the networking potential for this card down quite a bit. And since barely any motherboards have x4 slots, you’ll often need to use up an entire x16 slot for your working card – which seems like a bit of a waste.

A more recent PCIe gen 4 card, for example, could actually run in an x1 slot and still get the full bandwidth that is required for 10 Gigabit networking. Secondly these cards have a reputation for running REALLY hot – it’s well known that 10 Gigabit RJ45 Ethernet is hot and energy inefficient, but these older X540 cards in particular seem to have an issue with getting really hot and then dying. It’s great to find that out AFTER buying them, of course.

One good thing, though, is that these cards come with two RJ45 ports – meaning that I could hook an Ethernet cable up to BOTH of them, to a switch that supportslink aggregation, and get double the networking speeds. For example 2 Gigabit speeds on a 1 Gig switch, or 5 Gigabit speeds on a 2 and a half Gig switch – so that’s pretty cool.

But in general these cards aren’t really the best way to achieve 10 Gigabit networking – especially due to a REALLY ANNOYING issue they have out the box.

REALLY Noisy X540 Fans

When I first built my Homelab NAS, I noticed that it was NOISY. It was noisy enough that I started keeping my study door shut in the night, so that it wouldn’t keep everyone up. And through careful scientific research… which basically meant sticking my ear in the case when the PC was running… I concluded that the problem was coming from the CPU cooler. Blinking AMD stock coolers, they SUCK. So I started researching for alternative CPU coolers, but I decided to do a sound test one last time before hitting the checkout button – and I realised that it actually IS the networking card that is noisy. I initially ruled this out because I didn’t think that such a tiny device can actually be the cause of such a large noise nuisance, but yes, yes it was.

The fan was running at 100% max all the time, and I couldn’t easily see a way of controlling this (especially on Debian). So I… unplugged the fan. That mixed the noise issue – woohoo!

Some other X540 cards have metal heatsinks so I’ll probably look to switch to these passively cooled ones at some point in the future. For now though, I’m just going to ensure that I have an intake fan bringing cool air in past the card – which some people have done and had quite good success with, so I’m just going to stick with that for my HomeLab NAS.

Stick With RJ45?

That brings me onto a general issue when moving your networking above Gigabit speeds – HOW you should achieve this. I just personally blindly stayed with Ethernet – also called 10GBASE-T – because this is what I knew, and I already had dozens of Ethernet cables in my “bag of networking crap” as my wife affectionately calls it.

But it turns out that EVERYTHING about 10GBASE-T is hot and power hungry. It’s usually better to go with SFP+ switches and transceivers, since these are better equipped for very high networking speeds. However this is YET ANOTHER technology to actually learn about and support in a household, which is full of other tech anyway, and that’s not always practical – especially because while SFP+ switches ARE cheaper, the transceivers can add a big “unexpected” cost if you’re not careful.

Fibre, DAC or BASE-T?

This is where the CABLE choice matters a whole lot too. While copper Ethernet (BASE-T) does work, you WILL then need a transceiver module to use SFP+ switches and networking cards, and it’s probably not as good as fiber or DAC (Direct Attach Cable) for 10 Gigabit networking.

This is actually an entire other world so I’ll loop back to this in another video – otherwise I’d be talking for ages – but in hindsight I probably should have run DAC cable and purchased SFP+ switches and network cards – but lesson learned. I DO still have a working 10 Gig network (which I’ll show off in a minute), and that’s still pretty cool.

Which Switch to Buy?

When it actually came to buying the actual 10 Gig networking switch, I found the choices a bit overwhelming to be honest. That’s because I was used to cheap unmanaged switches from reputable companies like TP-Link and Netgear, which were (1) under £20, (2) super reliable (none have died on me yet) and (3) had really amazing reviews. A cheap Gigabit switch from a REPUTABLE company is often rock solid, and can deliver constant Gigabit speeds 24/7 with no problem.

But 10 Gig switches aren’t like this. So many of the reviews SEEMED good on the surface, but when you dug into them, a decently large number of customers had issues with dropping network speeds, needing to reset the switches every day, or simply the switches dying after a matter of months. And some of these switch companies seemed to have poor or nonexistent support.

I kinda started to conclude that there’s no point in trading in a super reliable, highly rated 1 Gigabit unmanaged switch for a less reliable (and potentially buggy) 2.5 or 10 Gigabit switch. Yes I might get faster throughput in some circumstances, especially when creating projects and importing all of the 4K footage, but everything else might be laggier due to having more bugs and less reliability. To combat this, I ended up paying a premium on a Ubiquiti switch, and I’ll loop back to this point in just a second but

My Workstation PC Is Too Small

another issue I faced (and it’s quite a big one) is that my main PC is micro ATX and the double slot graphics card completely BLOCKS the other PCIe slot – meaning that I couldn’t fit my hot and noisy 10 Gig networking card in (which was a pity). I tried LOTS of different things including brute forcing it in… which was a fairly stupid idea on my part (in hindsight) because it could have killed the PCIe slots or even the card… and I also checked out riser cables, but these were too bulky and rigid – and I couldn’t easily include them in my really small micro ATX case because my case had no real mounting positions for them – so that sucked, lesson learned.

In the end I gave up on installing this card in my workstation PC altogether, and instead I’m just using my motherboard’s 2 and a half Gig port. I WILL look to upgrade to an ATX motherboard and case at some point, but right now I’m in this weird half state – with a 10 Gigabit switch and Homelab NAS, but with nothing to FULLY take advantage of this.

What I Ended Up Doing

So at this point I should probably specifically discuss how I actually ended up “upgrading” to 10 Gigabit networking – and I guess I should say UPGRADED not “upgraded” because I DO havea 10 Gig network, albeit with only 1 device that fully takes advantage of it.

As I touched on earlier, I knew that I wanted a solid, reliable network switch – which meant going with a reputable company like Netgear, Cisco or Ubiquiti. Those first two are QUITE expensive, but luckily Ubiquiti has a suitable switch that is OFTEN mentioned on all the relevant subreddits,plus it’s fully managed too, meaning that I get a bunch of extra information, logging and features to control my upgraded home network… of two devices. But it’s a future proofed investment, I guess.

Yes I chose the Ubiquiti Flex XG switch that admittedly only has FOUR 10 Gigabit ports (and 1 Gigabit of uplink), but that’s good enough for me right now. And YES it cost me £250 which is more than I originally hoped to pay, but I’m still more than happy. I have had NO issues with connection speeds randomly dropping down, and I like that I can access extra management data via a browser – or even on my phone, it’s really cool actually.

Was It Worth It? Benchmarking Results

That’s all great then, but was it actually worth upgrading? I joked in the intro that I have saved myself 20 seconds each week when video editing, and that probably IS all the improvement I will see – RIGHT NOW (because I’ve stuck to 2 and a half Gig on my computer). But naturally this 10 Gig network is an INVESTMENT so that I can (1) support MORE devices in the future (streaming everything from my new NAS) and also potentially have (2) higher res or bitrate camera recordings without suddenly being bottlenecked by a Gigabit switch. So let’s drill into some results.

When I start off a new project, I will have footage from two of my cameras (plus separate audio). The cameras are currently set to record at 60 Megabits per second. When I drag these three files in, Premiere Pro will instantly start ingesting them and also creating “peak files” for them (so you can see the audio waveforms). And if I just drag these three files in, I can see in Task Manager that my networking speeds hit 1 and a half GigaBIT speeds at their peak, but admittedly this quickly settles down to sub-Gigabit speeds.

HOWEVER if I instead set my cameras to record at 100 Megabits per second and I drag THESE into a new project, my networking speeds do peak higher – sort of. They quickly hit 1.8 Gigabits per second, but then this also falls down to sub-Gigabit speeds – averaging around 700 megabits.

Now this is only when creating a new project, which I might only do once or twice a week. I ALSO have a bunch of extra footage – which is called broll – that will appear on screen as I talk. If I have a screen recording of a game and I drag this into my project, then it will peak higher because game recordings are often fairly bitrate – and I see network speeds hitting around 2 Gigabits per second based on multiple tests that I carried out.

Finally, let’s say that I’m super organized and I ALREADY have most of my broll when I go to create a new project – when I drag THOSE in, we can see that bandwidth usage is around 700 to 800 Megabits per second max over the network. This is for fairly low bitrate clips and static images, things like that, so it won’t stretch the network too much.

Now I will start off a project by dragging in my three “base” files, and dozens of broll files. Check out what happens in the networking tab as you can see it takes a bit of time to ingest everything, and network speeds are often near the Gigabit level – but they DO peak at 1.8 Gigabits or so a few times, which is interesting – at least to me.

Now you COULD argue that other than for a few seconds at a time, I don’t really NEED above Gigabit networking. But this is a one-off cost so while I probably DO only save 20 seconds a week right now,  at least now I have much more POTENTIAL, in terms of future proofing. So was it worth it? No. But did I have fun? Let me think… (Flashbacks) Also no. But at least now I can introduce myself at parties by saying “Hi I’m Tristan and I have 10 Gigabit networking via my Ubiquiti products!”. And that’s made it all worth it.

This video is part of a series I’m doing to upgrade my content creation workflow, and you cancheck out the other videos in the series by clicking somewhere here.

Tristan has been interested in computer hardware and software since he was 10 years old. He has built loads of computers over the years, along with installing, modifying and writing software (he’s a backend software developer ‘by trade’).Tristan also has an academic background in technology (in Math and Computer Science), so he enjoys drilling into the deeper aspects of technology.Tristan is also an avid PC gamer, with FFX and Rocket League being his favorite games.