On December 9, when the Apache Software Foundation disclosed a massive vulnerability in Log4j, its Java logging library, it triggered a cat-and-mouse game as IT professionals raced to secure their systems against cybercriminals looking to exploit a huge, now-known, issue. Among them were clients of George Glass, head of threat intelligence at governance and risk company Kroll. “Certain companies we spoke to knew there were applications that were impacted,” he says. The problem? They didn’t have access to them. “Maybe it’s a SaaS platform or it’s hosted somewhere else,” he says. They weren’t able to patch the Log4j binary itself, and instead faced a tricky decision: Turn off that specific application and stop using it, potentially refiguring their entire IT infrastructure, or take the risk that the third-party fix would come quicker than the state-sponsored and private hackers trying to take advantage.
At the same time as cybersecurity experts were trying to figure out their exposure to the problem, they were hit with successive warnings compelling them to act more quickly. First, the US Cybersecurity and Infrastructure Security Agency (CISA) set federal agencies a deadline of Christmas Eve to root out whether they used Log4j in their systems, and patch it. CISA director Jen Easterly said that it was the most serious vulnerability she’d seen in her career.
To help frazzled IT professionals understand whether they needed to do anything, CISA provided a five-step process, with three substeps, two verification methods, and a 12-part flow chart diagram with multiple routes and three outcomes (“vulnerable,” “not vulnerable,” and, confusingly, “likely not vulnerable”). As of early January, federal agencies had started work trying to identify any exposure to the Log4j vulnerability, but notably hadn’t fixed it entirely. A CISA spokesperson says “all large agencies have made significant progress.”
Then, on January 4, CISA and the Federal Trade Commission issued a warning to US businesses. “When vulnerabilities are discovered and exploited, it risks a loss or breach of personal information, financial loss, and other irreversible harms,” the FTC wrote. “It is critical that companies and their vendors relying on Log4j act now, in order to reduce the likelihood of harm to consumers, and to avoid FTC legal action.”
The federal body said it wouldn’t hesitate to use its full legal authority “to pursue companies that fail to take reasonable steps to protect consumer data from exposure as a result of Log4j, or similar known vulnerabilities in the future.”
The statement shifted the calculus of risk and liability for businesses. Threatened with legal action, they feel compelled to act. The challenge, though, is finding out whether they’re affected.
Log4j’s ubiquity makes it difficult to know whether any individual organization is affected. First discovered in Minecraft, the Log4j vulnerability has since been found in cloud applications, enterprise software, and on everyday web servers. The program is an event recorder, monitoring simple actions, both routine and errors, and reporting them to system administrators or users. And Log4j is one small but common component in tens of thousands of products—many of which are then bundled up into bigger projects. So-called indirect dependencies—packages or parts of programs that businesses use as part of their IT solution that unwittingly use Log4j—are one of the biggest risks, reckons Google, with more than four in five vulnerabilities hidden several layers deep into the interconnected web of software.
Most PopularThe End of Airbnb in New YorkBusiness
“The FTC has decided to swing a big hammer,” says Ian Thornton-Trump, chief information security officer at threat intelligence firm Cyjax. But he doesn’t necessarily think it’s the right move, calling it “impudent” and an unhelpful way of ramping up the situation. Large companies are conscious of what they need to do when confronted by such an issue, Thornton-Trump believes, and don’t need the FTC breathing down their neck to make them act. “What you don't need is a federal government agency telling you what the priorities are for your business when they don't even know what your actual business risk might be,” he says.
Others disagree. “Part of the chaos is that all of these big supply chain issues can cause a disjointed effort at remediation,” says Katie Moussouris, founder and CEO of Luta Security, a cybersecurity consultancy. “So I do think the FTC’s pressure is important.”
The FTC’s bravado in compelling companies to act is the end result of a government department wanting to genuinely help businesses in the United States and abroad but constrained by the lack of political will to push through meaningful cybersecurity legislation that isn’t focused on particular, limited areas, such as health care or financial data, says Thornton-Trump. As a result, US cybersecurity policy is reactive, trying to fix issues once they arrive under penalty of legal action, rather than proactive, he argues. Nevertheless, the FTC’s move is an important one: Though the FTC is to date the only government body globally to issue a warning to companies to fix the problem or else, the Log4j vulnerability affects hundreds of millions of devices.
Some businesses that fall under the regulator’s scope may have unexpected crises to deal with—for example, companies that have CCTV security cameras that are exposed to the internet with no compensating controls could find it “absolutely devastating,” says Thornton-Trump. Any internet-of-things devices that use Log4j and are vulnerable could act as an open door for hackers, easily allowing them access to a much bigger, more lucrative network through which they could wreak havoc. Thornton-Trump saw such an attempt happen at one of his clients, a managed service provider in Canada. “The firewall detected Log4j exploit attempts hitting CCTV cameras that were exposed,” he says. Thankfully, it was a security company scanning for vulnerabilities, and not a malicious attack.
It's unlikely that many businesses will be able to meet the FTC's demand to find and path the Log4j vulnerability immediately. Nor is it clear exactly how the FTC would be able to check if an organization was exposed to the Log4j vulnerability and hadn’t done anything, given how troublesome businesses are finding uncovering their own exposure. In fact, the FTC’s warning comes at a time when there’s a global shortage of cybersecurity professionals and work from home practices are putting more strain on the system than ever before, says Thornton-Trump. “They might not even have the capability to patch an update on this because their software that is vulnerable is out of lifecycle, or the developer has been sold.”
Most PopularThe End of Airbnb in New YorkBusiness
Such issues are likely to disproportionately affect small and medium businesses, he says—and make it nigh-on impossible to fix easily. Sonatype analysis has found that around 30 percent of the consumption of Log4j is from potentially vulnerable versions of the tool. “Some companies haven’t got the message, don’t have the materials, and don’t even know where to start,” says Fox. Sonatype is one of the companies that provide a scanning tool to identify the issue, if it exists. One client told them that without that, they’d have had to send out an email to 4,000 application owners they work with asking them to individually figure out if they were affected.
Part of the issue, of course, is the overreliance by for-profit businesses on open source, free software developed and maintained by a small, overstretched team of volunteers. Log4j’s issues aren’t the first—the Heartbleed bug that ravaged OpenSSL in 2014 is one high-profile example of a similar problem—and won’t be the last. “We wouldn’t buy products like cars or food from companies that had really terrible supply chain practices,” says Brian Fox, chief technology officer at Sonatype, a software supply chain management and security specialist. “Yet we’re doing it all the time with software.”
Companies who know they use Log4j and are on a fairly recent version of the utility have little to worry about and little to do. “That’s the unsexy answer to it: It actually can be very easy,” says Fox.
The problem emerges when companies don’t know they use Log4j, because it’s used in a small section of a brought-in application or tool they have no oversight over, and don’t know how to start looking for it. “It’s a bit like understanding what iron ore went into the steel that found its way into the piston in your car,” Glass says. “As a consumer, you have no chance of figuring that out.”
Log4j’s vulnerability, in a software library, makes it difficult to remedy, says Moussouris, because many organizations have to wait for the software providers to patch it themselves—something that can take time and testing. “Some organizations have higher technical skilled people inside of them that can work out different mitigations while they wait, but essentially, the majority of organizations rely on their vendors to produce high quality patches that include updated libraries or updated ingredients in those packages,” she says.
Yet companies big and small around the United States—and around the world—are having to move, and fast. One of them was Starling Bank, the UK-based challenger bank. Because its systems were largely built and coded in-house, they were able to detect quickly that their banking systems wouldn’t be affected by the Log4j vulnerability. “However, we also knew there might be potential vulnerabilities both in the third-party platforms that we use and in the library-originated code that we use to integrate them,” says Mark Rampton, the bank’s head of cybersecurity.
There were. “We quickly identified instances of Log4j code that were present in our third-party integrations that had been superseded by other logging frameworks,” he says. Starling removed those traces and prevented them from being used in the future. Simultaneously, the bank tasked its security operation center (SOC) with analyzing hundreds of thousands of events to see if Starling was being targeted by those looking for Log4j vulnerabilities. They weren’t, but are keeping an eye out. The efforts required are significant, but necessary, says Rampton. “We decided to take a ‘guilty until proven innocent’ approach, as the vulnerability was unravelling at such a pace that we couldn’t make any assumptions,” he says.
“I get where the FTC are trying to come from,” says Thornton-Trump. “They’re trying to encourage people to do vulnerability management. But it’s absolutely tone deaf to the actual threat risk that this vulnerability poses to many businesses. They’re basically making you press the panic button on something you don’t even know if you have at this point.”
More Great WIRED Stories📩 The latest on tech, science, and more: Get our newsletters!The race to find “green” heliumCovid will become endemic. What happens now?A year in, Biden's China policy looks a lot like Trump'sThe 18 TV shows we’re looking forward to in 202How to guard against smishing attacks👁️ Explore AI like never before with our new database📱 Torn between the latest phones? Never fear—check out our iPhone buying guide and favorite Android phones