rbowes-r7 (95)

On October 10, 2023, Citrix posted an advisory about a high-risk vulnerability in Citrix ADC, which affects the following versions:

• NetScaler ADC and NetScaler Gateway 14.1 before 14.1-8.50
  
• NetScaler ADC and NetScaler Gateway 13.1 before 13.1-49.15
  
• NetScaler ADC and NetScaler Gateway 13.0 before 13.0-92.19
  
• NetScaler ADC 13.1-FIPS before 13.1-37.164
  
• NetScaler ADC 12.1-FIPS before 12.1-55.300
  
• NetScaler ADC 12.1-NDcPP before 12.1-55.300
  

Assetnote coined this as Citrixbleed when they released their research on it. Basically, it lets you steal session tokens (and other data from memory). It’s been added to KEV based on Citrix’s statement.

The vulnerability itself is subtle, and is due to misusing a C API function (as so many vulns are). The following disassembled code is used as part of the OpenID response:

iVar3 = snprintf(print_temp_rule,0x20000, "{\"issuer\": \"https://%.*s\", \"authorization_endpoint\": \"https://%.*s/oauth/ idp/login\", \"token_endpoint\": \"https://%.*s/oauth/idp/token\", \"jwks_uri\": \"https://%.*s/oauth/idp/certs\", \"response_types_supported\": [\"code\", \"toke n\", \"id_token\"], \"id_token_signing_alg_values_supported\": [\"RS256\"], \"end _session_endpoint\": \"https://%.*s/oauth/idp/logout\", \"frontchannel_logout_sup ported\": true, \"scopes_supported\": [\"openid\", \"ctxs_cc\"], \"claims_support ed\": [\"sub\", \"iss\", \"aud\", \"exp\", \"iat\", \"auth_time\", \"acr\", \"amr \", \"email\", \"given_name\", \"family_name\", \"nickname\"], \"userinfo_endpoin t\": \"https://%.*s/oauth/idp/userinfo\", \"subject_types_supported\": [\"public\"]}" ,uVar5,pbVar8,uVar5,pbVar8,uVar5,pbVar8,uVar5,pbVar8,uVar5,pbVar8,uVar5,pbVar8); authv2_json_resp = 1; iVar3 = ns_vpn_send_response(param_1,0x100040,print_temp_rule,iVar3);

It uses the snprintf() function, which returns the number of bytes the call wants to write to the buffer, not the number of bytes it actually writes to the buffer; from the manpage printf(3):

The functions snprintf() and vsnprintf() do not write more than size bytes (including the terminating null byte (‘\0’)). If the output was truncated due to this limit, then the return value is the number of characters (excluding the terminating null byte) which would have been written to the final string if enough space had been available. Thus, a return value of size or more means that the output was truncated

That length value – which can be longer than the size of the buffer – is used when sending a reply. That means that if the desired string is more than 0x20000 bytes long, it will respond with memory after the end of the buffer, which includes session tokens.

The work done by watchTowr and later VulnCheck is super cool, and outlines different great ways to exploit the vulnerability (we based the Rapid7 Analysis on watchTowr’s). Neither of them mention something sorta-important: all the known exploits land you in a super-restrictive BSD jail with no meaningful OS access.

The application running in the jail has access to configure the system, so presumably there’s a path from the jail to the OS, but to my knowledge nobody has done much yet. In our analysis, we showed how you could steal an admin session then change the system’s password, but surely there are better avenues waiting to be found!

This was released in an advisory with 15 different vulnerabilities! I chained together 4 of them (or 5, depending on how you count) to get RCE. Weaponizing was tricky due to aggressive filters, but we figured out how to execute any arbitrary command with encoding on both Windows and Linux.

Note that the analysis before is for a separate issue – either a silently-patched vuln or some cleanup. We’ve posted a Rapid7 Analysis with the full details, so check that out!

I spent some time this week looking at the advisory for CVE-2023-3519, and the associated patch (our blog post). To test, I installed versions 13.0-91.12 and 13.0-91.13, and did a bunch of diffing / patch analysis to try and understand the vulnerability (I chose a 13.0 version with the idea that there will be less active maintenance, and therefore less other changes to the codebase). Ultimately I hit a paywall in the trial version, so I figured I’d post what I have and move onto something else.

If I’m right, the issue appears to be memory corruption in the SAML parser, in the nsppe process. Since nsppe runs as root, successful exploitation would likely lead to code execution as root (it would require network access to the management port).

This has been added to the known exploited vulnerabilities list, but they identify it as a “code injection” attack, so it’s possible that this isn’t right.

After installing, I diffed everything on the filesystem to look for major changes. There were no obvious changes in the scripting stuff (PHP/Perl/etc), which is where I kinda expected to see the patch. Next, I diffed the binaries under /netscaler and other folders using this neat little Bash command:

$ for i in $(cat binaries.txt); do diff -rub <(strings -n8 ./13.0-91.12/$i | sort) <(strings -n8 ./13.0-91.13/$i | sort); done

That command takes a list of binaries from the file binaries.txt (which I generated with find -type f), runs strings on both versions, then diffs the strings list. It’s a nice quick way to quickly find or/ triage the files that are most interesting, though it won’t find subtle logic changes.

Most of the changes are just version numbers or other simple things that we can immediately discount. But one file, nsppe, had interesting looking changes, such as new error messages about a length value:

 aaa_rpc_auth_fail_free_aaa_info
 aaa_rpc_auth_rejected
+AAA SAMLIDP ASSERTION REQUEST: List of Canonicalization Method exceeds max limit <%d>, current value <%d>
+AAA SAMLIDP ASSERTION REQUEST: List of Transforms Method exceeds max limit <%d>, current value <%d>
 aaa_samlidp_tot_authnreq_fail
 aaa_samlidp_tot_authnreq_succ

I grabbed the old and new versions of the nsppe binary, which it turns out is the NetScaler Packet Parsing Engine, and disassembled them. The binary is a beast, but we found some documentation online that explains its purpose.

Thankfully, the changes made by this patch are limited. In case you want to make sure we’re on the same page, here are the sha256es of the two versions I tested:

$ sha256 nsppe-13.0-91.12
SHA256 (nsppe-13.0-91.12) = 9d1b39e59374088f355a9b26a1b9bb94addfb1dcd7af7fd28acd853403b414f7

$ sha256 nsppe-13.0-91.13
SHA256 (nsppe-13.0-91.13) = 3779d1148df674e3f107b384f1e770e658128a6d603399c3d595c31a0d37af4e

I diffed the patches using a complicated series of sed statements and the diff CLI tool to determine where the majority of the changes are. For the most part, the patch adds eight different length checks, each of which compares a local variable to 9, then prints an error message if it’s larger. I put together this list of functions, along with the address where a check was added and the error message if the check fails (these are from the 91.13 version of /netscaler/nsppe):

• Function: ns_aaa_saml_parse_authn_request():
  
  • 0xCA920B – AAA SAMLIDP ASSERTION REQUEST: List of Canonicalization Method exceeds max limit <%d>, current value <%d>
    
  • 0xCAA57E – AAA SAMLIDP ASSERTION REQUEST: List of Transforms Method exceeds max limit <%d>, current value <%d>
    
• Function ns_aaa_saml_parse_logout_response():
  
  • 0xCB09A6 – AAA SAML LOGOUT RESPONSE: List of Canonicalization Method exceeds max limit <%d>, current value <%d>
    
  • 0xCB1E29 – AAA SAML LOGOUT RESPONSE: List of Transforms Method exceeds max limit <%d>, current value <%d>
    
• Function ns_aaa_saml_parse_logout_request():
  
  • 0xCB8154 – AAA SAML LOGOUT REQUEST: List of Canonicalization Method exceeds max limit <%d>, current value <%d>
    
  • 0xCB94DB – AAA SAML LOGOUT REQUEST: List of Transforms Method exceeds max limit <%d>, current value <%d>
    
• Function ns_aaa_saml_parse_assertion():
  
  • 0xD038E7 – AAA SAMLSP ASSERTION RESPONSE: List of Canonicalization Method exceeds max limit <%d>, current value <%d>
    
  • 0xD04CD3 – AAA SAMLSP ASSERTION RESPONSE: List of Transforms Method exceeds max limit <%d>, current value <%d>
    

As you can see, the error messages are very similar, and strongly point to a length check being added. Based on the error messages, I would speculate that the vulnerability is triggered by sending too many canonicalization or transform methods in a SAML message. I randomly picked one of those errors (the Canonicalization Method check from ns_aaa_saml_parse_authn_request), and looked at how the newly-validated value is used. The number_of_canonicalizations value checked at 0xCA920B (in 91.13 – the patched version) is used to write into an array that I called array_indexed_into:

.text:0000000000CA9AAB loc_CA9AAB:                             ; CODE XREF: ns_aaa_saml_parse_authn_request+5C68↑j
.text:0000000000CA9AAB                                         ; ns_aaa_saml_parse_authn_request+5C7D↑j
.text:0000000000CA9AAB                 mov     eax, [rbp+number_of_canonicalizations]
.text:0000000000CA9AAE                 mov     rdx, [rbp+array_indexed_into]
.text:0000000000CA9AB5                 mov     dword ptr [rdx+rax*4+40h], 2
.text:0000000000CA9ABD                 jmp     short loc_CA9AD0

Tracing the targeted array backwards across several function calls leads to:

.text:00000000008128E9                 mov     edx, 35h ; '5'
.text:00000000008128EE                 mov     esi, 210h
.text:00000000008128F3                 mov     edi, offset ns_alloc_memz
.text:00000000008128F8                 call    ns_meminst_alloc

ns_meminst_alloc and ns_alloc_memz are memory-allocation functions, and it looks like heap memory, though I’m not 100% sure on that one.

So anyway, my guess is that the core of this vulnerability is a probably-heap overflow when parsing certain types of SAML requests with too many canonicalization or transform methods.

I tried to verify that, but I believe the AAA features are behind a paywall (licensewall?), so I can’t access them:

> enable ns feature AAA
ERROR: Feature(s) not licensed

Rather than fuss with licensing, I figured I’d post what I have and carry on!

This vulnerability is trivial to exploit, particularly with proofs of concept (and a Metasploit PR) in the works.

The saving grace is that this doesn’t appear to to be particularly common on the internet.. different Shodan queries show like 5-6 instances facing the internet.

If you do run this, though, patch ASAP! It’s very easy remote root against the server.

A special request to the Fortra Globalscape admin service returns what they call a TER – Trial Extension Request – which is a lightly-obfuscated Base64 string that contains the SHA256 sum of the harddrive’s serial number. The search space is small enough that it’s possible to reasonably bruteforce the hash, even on laptop hardware

This is effectively a zipbomb – the inbound message parser for the Fortra Globalscape’s admin protocol accepts compressed messages, but the decompression code is recursive. That means that, if you send a message that decompresses into itself with a short header (effectively a “quine”), it will recurse infinitely and eventually blow the stack.

While it doesn’t give any useful access, it’s a neat vuln with a (mathematically) neat exploit!

PoC

At its core, this is a blind out of bounds memory read in the packet parser for the Fortra Globalscape admin server. That means that you can trick the packet parser into leaving the memory bounds of its packet and processing other memory as if it was part of the packet. The most likely result is sending the parser into unallocated memory and crashing the process.

However, if an attacker can correctly guess the offset of another user’s login packet (which isn’t as crazy as it sounds – packets are typically close together on the heap), they can use the other user’s login and bypass authentication altogether.

Without some sorta heap massaging or very lucky guessing, this is going to be very difficult to exploit, but it’s definitely interesting!

Simple PoC

This issue requires a fairly specific configuration to work. First, Apache has to be configured in a pretty specific way – a RewriteRule that passes user-controlled data into a query string. This line, from our configuration file above, is exploitable, as the $1 (user data) is after the ? (start of the query string):

RewriteRule "^/(.*)" "http://localhost:8081/?arg=$1" [P]

Whereas even a small change will break the exploit; this does not work:

RewriteRule "^/(.*)" "http://localhost:8081/$1" [P]

Additionally, this must be a security boundary — that is, smuggling an HTTP request must bypass some sort of access control check. We put together a Github search that tries to find software that would be exploitable if it was running on a vulnerable version of Apache, but nothing stands out as particularly interesting.

Applications and devices that come as complete images with multiple servers (such as a lot of enterprise software) would probably be better targets, but checking each one is difficult. We looked into this as an alternative way to exploit CVE-2022-1388, since the core of that vulnerability involved bypassing a reverse proxy’s security checks, but its configuration is not vulnerable to CVE-2023-25690.

The final place we’d expect to see vulnerabilities is in reverse proxies that are configured to enforce some kind of ACL check or filtering, and that also do URL rewriting. One would have to test the specific configuration and application(s) that are being proxied; there’s no easy way to scan for this sort of issue.

While we are not aware of any specific applications that are vulnerable to this, they may turn up as people investigate this vulnerability more.

This lives on the network perimeter and uses laughably old versions of software (like Ruby 1.9.3). I had more trouble preventing it from crashing than actually exploiting it. This also tends to store privileged information, since it’s a secure file transfer service. Kinda really bad.

This is currently unpatched and vulnerable in the default state. The time from reading the mitigation to having a working exploit was less than day, and that’s for somebody who isn’t super good at Java vulnerabilities.

This is a sorta complicated collection of vulnerabilities. In the Rapid7 analysis, we focused on the two most popular pieces of software, which run an ancient version of Santuario 1.4.1 (in part because the public PoCs target that version). Santuario 1.4.1 has absolutely trivial RCE issues.

But, according to the disclosure blog, other versions have more recent versions of Santuario, but run a version of Xalan that’s vulnerable to a whole other issue – CVE-2014-0107. Dinh found a novel way to exploit CVE-2014-0107, which is super cool, but it was overshadowed by the much simpler vuln in Santuario. I wish they’d named the specific software that CVE-2014-0107 worked against – that’d be a neat one to test, but I don’t think their blog is specific.

The saving grace here is that none of these projects are vulnerable unless SAML is enabled (although in some cases, they’re vulnerable forever if SAML is enabled once).

An exploit is published on the Full Disclosure mailing list, and it’s been exploited in the wild.

According to the vendor advisory, this is an authentication bypass in SugarApplication.php:

The underlying vulnerability relies on a missing authentication check in the loadUser() method in include/MVC/SugarApplication.php

Unfortunately, I couldn’t get ahold of a current version of the software to test. I got the most recent community version running (it’s quite old), but while that file exists, the auth bypass doesn’t seem to work.

Because this is CSRF, it’s very difficult to exploit – it requires an attacker to know the network layout, then either get super lucky or socially engineer an administrator. Although it’s an interesting vulnerability with a great payoff, I’ll be surprised if this gets exploited.

On the flip side, the PoCs I wrote create a persistent backdoor, and don’t require the attacker to have direct network access to the management interface – it can all be done through the administrator’s browser.

This permits a user who already has an administrator account to create a shell on the target device. There are other (authorized) ways to do this, so this vulnerability is pretty minor, IMO, although if it IS exploited it grants access to a network load balancer which is kind of a big deal.

A vulnerability lets you send requests to the backend API service that appear to be coming from a trusted frontend application. As a result, you can call any REST API without authentication, which is pretty bad considering this is a security appliance.

This is I think the 6th major issue with Zimbra this year. It’s not really their fault, they use Amavis which uses cpio which is vulnerable to CVE-2015-1197, but the attack surface for incoming emails is HUGE.

Not to mention, this is one of several vulnerabilities this year that was being exploited in the wild before being discovered, which means Zimbra is an active target for the Bad Guys.

If you’re still using Zimbra, you might want to seriously reconsider. I betcha there are others, and they’re probably being exploited.