Security Advisory: Remote Code Execution Vulnerability (CVE-2023-3519)

Introduction

On 18 July 2023, Citrix published a security advisory that addressed a critical vulnerability with CVSS score 9.8 for CVE-2023-3519 for RCE (Remote Code Execution) in NetScaler ADC (formerly known as Citrix ADC) and NetScaler Gateway (formerly known as Citrix Gateway). 

This vulnerability created a lot of buzz in the last several days. Many reports claim that the vulnerability was being exploited as a zero-day attack in the wild as threat actors were dropping a web shell onto organization's critical infrastructure.

The advisory from Citrix also includes additional vulnerabilities affecting NetScaler users.

• CVE-2023-3466 - A reflected Cross-Site Scripting (XSS) vulnerability which requires the victim to access an attacker-controlled link in the browser while being on a network with connectivity to the NSIP. NetScaler IP (NSIP) address is the IP address where you access the NetScaler for management purposes.
• CVE-2023-3467 - A Privilege Escalation vulnerability that requires authenticated access to NSIP or SNIP with management interface access. A subnet IP address (SNIP) is a NetScaler owned IP address that is used by NetScaler to communicate with the servers.

Key Takeaways 

• New Vulnerability Discovered: A new vulnerability, CVE-2023-3519, has been discovered that impacts NetScaler ADC NetScaler Gateway applications. 
• Upgrade Your Application: If you see your ADC or Gateway version below, Zscaler strongly urges you to upgrade to a safe version.
  • NetScaler ADC and NetScaler Gateway 13.1 before 13.1-49.13 
  • NetScaler ADC and NetScaler Gateway 13.0 before 13.0-91.13 
  • NetScaler ADC 13.1-FIPS before 13.1-37.159
  • NetScaler ADC 12.1-FIPS before 12.1-55.297
  • NetScaler ADC 12.1-NDcPP before 12.1-55.297

Possible Execution/PoC

CVE-2023-3519 could allow an unauthenticated threat actor to trigger a stack buffer overflow in the NetScaler Packet Processing Engine (nsppe) process by sending a specially crafted HTTP GET request. Since the nsppe runs as root, successful exploitation would likely result in arbitrary code execution as the ‘root’.

This is an example of the HTTP request with GET method:

_{Figure 1: Example Packet Capture with shell code}

A proof-of-concept (PoC) for CVE-2023-3519 in Citrix ADC can be found on GitHub.

Attack Chain

A threat actor can exploit CVE-2023-3519 by uploading files containing malicious web shells and scripts, allowing them to scan networks and extract sensitive information. 

A server's configuration files contain encrypted passwords that can be viewed and decrypted by decryption keys that are available on the same server.  Thus, making configuration files an obvious target for threat actors. By decrypting Active Directory credentials, a threat actor can retrieve a wide range of information, including:

• details about users
• computers
• groups
• subnets
• organizational units

_{Figure 2: Attack chain of Citrix Gateway CVE-2023-3519 unauthenticated remote code execution}

1. Initial Access - According to CISA, a threat actor can upload a TGZ File (a compressed archive created using GZIP) containing a generic web shell, discovery script, and setuid binary file on the Citrix's NetScaler Application Delivery Controller (ADC) application. Through the web shell, the threat actor can execute remote commands on the compromised system, establishing a reliable command and control channel.

2. Privilege Escalation - The uploaded TGZ file consists of a setuid binary file which is used by threat actors to exploit Elevation Control Mechanism and gain elevated permissions on a system.

3. Credential Access - A threat actor uses the NetScaler configuration files, located in /flash/nsconfig/keys/updated/* and /nsconfig/ns.conf, to find an encrypted password that can be decrypted by the key stored on the ADC application. Using these keys, the Active Directory credentials are decrypted from the configuration file.

4. Discovery - The threat actor uses the newly acquired, decrypted credentials to query the Active Directory for trusted domains, organization units (OU), computers, users, etc. within the network that can be exploited to move laterally or escalate privileges.  

5. Collection - The threat actor uses the 'tarball' command to compress the collected data and uses 'openssl' to encrypt the data. The following command is used by the threat actor to collect the compromised data from the infected system:

tar -czvf - /var/tmp/all.txt | openssl des3 -salt -k <> -out /var/tmp/test.tar.gz

6. Defense Evasion - Exfiltrated, collected data can be uploaded as an image file to a web accessible path to bypass detection engines using this command:

cp /var/tmp/test.tar.gz /netscaler/ns_gui/vpn/medialogininit.png.

As per public reports, a segmented environment where ADC applications were deployed prevented the threat actors from discovering critical infrastructure. The threat actors attempted to:

• Execute a subnet-wide curl command to identify what was accessible from within the network as well as potential lateral movement targets.
• Verify outbound network connectivity with a ping command (ping -c 1 google.com).
• Execute host commands for a subnet-wide DNS lookup.

Like the reports state, the threat actors also attempted to delete their artifacts. They deleted the authorization configuration file (/etc/auth.conf) to prevent configured users (e.g., admin) from logging in remotely (e.g., CLI). Normally, an organization would need to reboot into single user mode (which may have deleted artifacts from the device) to regain access to the ADC application. However, in this case, the victim used an a readily available SSH key to gain access into the appliance without rebooting.

The threat actor’s post-exploitation lateral movement attempts were also blocked by network-segmentation controls. The threat actors implanted a second web shell on the victim that they later removed. This was likely a PHP shell with proxying capability. The threat actors likely used this to attempt proxying SMB traffic to the DC (the victim observed SMB connections where the actors attempted to use the previously decrypted AD credential to authenticate with the DC from the ADC via a virtual machine). Firewall and account restrictions (only certain internal accounts could authenticate to the DC) blocked this activity.

^{Figure 3: Metasploit releases the exploit for CVE-2023-3519}

Affected Products

The following supported versions of NetScaler ADC and NetScaler Gateway are affected by the vulnerabilities: 

• NetScaler ADC and NetScaler Gateway 13.1 before 13.1-49.13 
• NetScaler ADC and NetScaler Gateway 13.0 before 13.0-91.13 
• NetScaler ADC 13.1-FIPS before 13.1-37.159
• NetScaler ADC 12.1-FIPS before 12.1-55.297
• NetScaler ADC 12.1-NDcPP before 12.1-55.297

Mitigations

Zscaler has observed exploits of CVE-2023-3519 on unmitigated applications. Zscaler strongly urges affected customers of NetScaler ADC and NetScaler Gateway to install the relevant updated versions as soon as possible. 

• NetScaler ADC and NetScaler Gateway 13.1-49.13  and later releases
• NetScaler ADC and NetScaler Gateway 13.0-91.13  and later releases of 13.0  
• NetScaler ADC 13.1-FIPS 13.1-37.159 and later releases of 13.1-FIPS  
• NetScaler ADC 12.1-FIPS 12.1-55.297 and later releases of 12.1-FIPS  
• NetScaler ADC 12.1-NDcPP 12.1-55.297 and later releases of 12.1-NDcPP 

Zscaler Coverage

Zscaler’s ThreatLabZ team has deployed protection.

• Zscaler Advanced Threat Protection:
  1. App.Exploit.CVE-2023-3519
• Zscaler Private Access Protection:
  1. Remote Code Execution : 932100 (Remote Command Execution: Unix Command Injection basic coverage)     
  2. Remote Code Execution : 932200 (RCE Bypass Technique)     
  3. PHP Injection : 933100 (PHP Injection Attack: PHP Open Tag Found)

We will update this article as information becomes available or if additional protection is put in place by Zscaler. 

Resources

• Citrix ADC and Citrix Gateway Security Bulletin for CVE-2023-3519, CVE-2023-3466, CVE-2023-346
• Threat Actors Exploiting Citrix CVE-2023-3519 to Implant Webshells | CISA
• GitHub - BishopFox/CVE-2023-3519: RCE exploit for CVE-2023-3519
• Hundreds of Citrix NetScaler ADC and Gateway Servers Hacked in Major Cyber Attack (thehackernews.com)
• https://nvd.nist.gov/vuln/detail/CVE-2023-3519
• https://blog.assetnote.io/2023/07/24/citrix-rce-part-2-cve-2023-3519/