The unexpected experience
A while ago, I sat beside a ServiceNow, Inc. Consultant. He was able, without a local user, to login into a self-hosted ServiceNow® Instance which had no connection to the outside world. We were on-site at the Customer’s offices and he had to run some scripts that can only be run with a special role — the maint role — which is reserved to ServiceNow employees.
Seeing this ServiceNow employee without a local user being on the customer instance without a “true” user but with higher access rights than the admin/security_admin made me think, how secure ServiceNow would be. Is there really a backdoor?
Logins for ServiceNow employees
As ServiceNow is a growing company and support had to be given without thousands of ServiceNow-own users on customer instances, a way had to be find to allow access for employees to give support even if the customers locked out themselves.
This is a relatively tricky task as you don‘t want to give all employees access to all instances with the same password and you want the passwords to be impermanent.
Therefor ServiceNow created a mechanism which depends on an asymmetrical cryptographic method. All Instances have a public key which can be replaced via patches or upgrades and ServiceNow has a securely stored private key.
The encrypted text (or to be precise the cryptographic hash of that text) itself is the password, the to be encrypted text consists of a generic username specific for the ServiceNow employee, customer instance specific data like a unique ID of the instance, the build and a normalized timestamp.
Encryption is done by a central instance which is only accessible from the ServiceNow-own network only. Employees therefor must have connection to the company‘s network, e.g. through VPN, to generate their password.
A secure backdoor
As described above, the backdoor uses person, instance and time specific data to make sure that just this and only this employee can log into one specific instance for a certain time only. The standard validity time for a password is 4 hours from generation on.
The question is, if the used cryptograophic function is safe.
ServiceNow currently uses a 224-bit private key[1] and uses SHA256-ECDSA to generate the hashes/passwords (HMACs).
As SHA256-ECDSA is an Elliptic Curve algorithm, the 224-bit private key is as safe as a 2048-bit standard asymmetric key according to the OpenSSL Wiki.
Convenient Restricted Access
While the mechanism was in development (which it surely still is), ServiceNow recognised that access had to be restricted to employees having a current case with the customer. Therefor, for normal support staff, generating a temporary password is possible only if there is an active incident, problem or change referring the exact instance a password should be created for and if the employee is assigned to that task. There‘s just a handful of employees (e.g. monitoring staff) that can generate passwords without such a case.
The password generation is automated as much as possible: On the incident/problem/change form a “Hop” button is provided, forwarding the employee to the internal instance generating the password, which itself forwards the employee to the customers instance automatically logging in the employee using HTTP URL parameters. No password has to be copied manually.
The central instance generating the password also is secured using two-factor authentication for all employees.
For offline instances (self-hosted instances without a connection to ServiceNow) like the one mentioned in the beginning, the password is generated but still is to be copied manually to log in.
To be sure ServiceNow even published a policy internally regulating access with generated passwords, e.g. stating that access to the customers instance must always be consented by the customer itself.
Access regardless of IP Address Restrictions setup
As support has to help even if the customer has been locked out by unlucky circumstances — like a wrongly setup IP address restriction — ServiceNow had to be sure that all ServiceNow-owned IP addresses, incl. the company-internal network, have access to the instances. Therefor a hidden whitelist in the distribution/deployment files exists, always enhancing a customer‘s IP address whitelist, if one is setup.
These IP addresses are:
127.0.0.1 |
4.28.176.51 |
So no matter what IP address restrictions you are configuring in your instances, those IP addresses are always added to that whitelist without your notice — giving ServiceNow access to your instance, while above mentioned restricted access still is in place of course.
Hardening / securing your instance
Besides the ServiceNow instance hardening customer security document (KB0550654) you might want to implement the following tips to restrict ServiceNow access to your instance as much as possible.
Set temporary passwords’ max validity to zero
The validity of the temporary passwords can be configured, so that an employee can login only within 2 instead of 4 hours – effectively making the temporary passwords valid only for 2 hours.
As you can configure the maximum number of hours that a temporary password is valid, you can also bring it down to zero (0) hours, effectively locking out ServiceNow, Inc. employees.
To configure this number, create a System Property called “glide.security.credential.max_hours” and set its value to the amount of hours you like the temporary passwords to be valid.
Beware: In case of support, this might obstruct ServiceNow, Inc. employees resulting in longer support times!
Beware: Choosing the values 1, 2 and 3 won’t lock out support personnel, BUT will lead to unexpected behavior for the support personnel, because the generated password will be temporarily valid in the future as it is generated for a time point 4 hours ahead.
A better way to permanently lock-out ServiceNow, Inc. support staff is enabling the Plugin com.snc.snc_access_control (see below), as it gives the possibility to allow access for a list of specific persons only and for a stricter time limit.
Enable Plugin com.snc.snc_access_control manually
You can enable the mentioned plugin yourself using following code:
// activating the plugin
var worker = new GlidePluginManagerWorker();
worker.setProgressName('com.snc.snc_access_control activation');
worker.setPluginId('com.snc.snc_access_control');
worker.setBackground(true);
worker.start();
gs.log('PluginManagerWorker: '+ worker.getProgressID());
The progress can be checked under System Diagnostics → Progress Workers.
Enabling this plugin installs a table and loads an Authentication Gate into your system, which checks, if a user is an ServiceNow, Inc. employee or a normal user. Normal user just pass while ServiceNow, Inc. logins are checked against the installed table.
If there is a valid list in the table where the user is contained in or a valid entry with an asterisk, then the access is granted for the ServiceNow, Inc. employee as well.
To configure, see the docs under: https://docs.servicenow.com/bundle/kingston-platform-administration/page/administer/security/task/t_ConfigureAccessControl.html
Authentication Gates truely secure your system as they are evaluated before the actual authentication process, which checks passwords against local users or temporary credentials. Therefor, if an authentication gate fails, the whole login process is canceled.
Unfortunately, Authentication Gates are not further documented by ServiceNow. They are used in MSP scenarios for example. Maybe I will give a short introduction and a brief documentation to Authentication Gates in a future article.
| Update 2018-10-30:
After a deeper analysis of the Authentication Gates, there is one exception to the ServiceNow-own users, that are gated by this solution: the user “localhost@snc” is never checked by Authentication Gates (directly bypassing them), being always directly authenticated using the backdoor mechanism! For on-premise customers: to completely disable ServiceNow, Inc. employee access, you can override the public key by a self-generated one without corrupting the product itself. As a few steps are involved for this and there is no guide yet, please contact me for further information. |
More Restricted IP-Access
Create a System Property “glide.ip.authenticate.strict”, type “true | false” and value “true”. This should reduce the whitelisted IPs to a minimum of:
127.0.0.1,10.0.0.0/8
199.91.136.0/21
103.23.64.0/22
37.98.232.0/21
149.96.0.0/17
This should be a “list of IPs that just includes the data center, secure VPN (but not SN corporate/guest IPs)”.
Attention: double-check effect of this together with an employee from ServiceNow, Inc. by watching him trying to access your instance and showing all his IP addresses to you.
Restrict viewing of stats.do
Set the value of the System Property “glide.security.diag_txns_acl” to “true”.
History on that backdoor
Nowadays the system all in all is very secure as described above. This wasn‘t the case all the time. When I first started to take the ServiceNow® platform appart to check it‘s security, this backdoor was secured by a custom rsa-like algorithm using a 40-bit private key to generate passwords with 5 “printable characters”.
This is against all best practices when implementing encryption-enabled security mechanisms. The justification was quite scary: the passwords shouldn‘t be too long for the sake of comfort of the personnel, ServiceNow wouldn‘t be a top target for hackers and if the private key was hacked it could simply be exchanged. ServiceNow’s own assumption was that the private key could be breached within a few weeks or few months.
But, ServiceNow has done their homeworks looking at the current state of the backdoor.
Footnotes
[1] By using the key-pair generator incorporated in the MID Server package, a public key with the same representational length could only be reconstructed when using 224-bit length for key generation.
One comment
Comments are closed.