DansGuardian Documentation Wiki

You are here: Main Index » using_ntlm_for_user_identification


|

Wiki Information

Using NTLM for User Identification

If you wish to authenticate your users to an NT/Active Directory domain (implemented either with Microsoft products or with OSS replacements such as Samba) using NTLM, here is some detailed information and some instructions that may help you. (You might also wish to authenticate your users to an Active Directory domain using LDAP or Kerberos but not NTLM.)

NTLM has been very popular amongst DansGuardian administrators, perhaps because it (was?) “the Microsoft way”. Yet viewed from a less vendor-centric perspective, NTLM's chief (and perhaps only significant) advantage is that users usually don't have to retype any password. Compared to Ident authorization (that provides similar benefits), it's much more secure, and it's fully supported more often.

(Note that you may no longer have the option of using NTLM authentication if you chose the “transparent-intercepting” configuration family rather than the “explicit-proxy” configuration family for your DansGuardian. You made the choice of which configuration family to use when first setting up your DansGuardian. For more details see Two Configuration Families.)

Definition

NTLM is a Microsoft automatic challenge-response authentication method, often associated with either Internet Explorer or Samba. Although no longer used by Microsoft for remote file access (SMB/CIFS), it's still one of the options Microsoft can “negotiate” for browsing the web.

It's closely integrated into the Microsoft Windows OS, and Microsoft web browers include explicit support for it.

It reuses the HTTP authentication headers and return codes, even though it's not really an HTTP authenticaton scheme. Unlike most web auth schemes, it authenticates connections (not requests).

Required Environment

With NTLM, the browser supplies user credentials by in essence automatically reusing the password entered when the user logged on, without ever prompting the user to enter another password. This functionality is often not fully supported or not supported by default by non-Microsoft browsers, particularly web browsers that are explicitly platform-agnostic. As a result, NTLM may be most appropriate in a homogeneous browser environment (but see next paragraph).

Minimizing Authentication Hassles should be possible, even with heterogeneous browsers and even with heterogeneous OSs. However doing so is highly environment- and version-dependent, and will probably require some detailed tweaking in each case. Minimizing authentication hassles may or may not mean entirely eliminating them by using automatic authentication everywhere all the time. And it may or may not mean using NTLM everywhere. (Note the important distinction between behavior [automatic authentication] and mechanism [NTLM].)

For historical reasons OSS NTLM is closely associated with the Samba software application. Examples of how to set up NTLM explicitly reference Samba. However in actuality neither Samba as the main AD controller nor sharing files via Samba are required to use NTLM with DansGuardian/Squid. To provide NTLM authentication even though Samba is not the main AD controller, just install Samba as a subsidiary server (possibly a BDC or even just a file server, preferably unused except for auth [NTLM, logon, etc.]), then follow all the directions below just as though your primary domain controller were a Samba system. The web browser will talk to DansGuardian/Squid, which will talk via a Squid helper program to the stub Samba, which in turn will talk to the rest of the (largely non-Samba) AD system.

(To provide NTLM authentication without using Samba at all by using a modified configuration and an authentication helper program that's not specific to Samba seems possible in theory. But the only known available implementation of anything similar relies on an “LDAP” Squid helper program talking directly to AD and transmitting your domain account usernames and passwords over the network in cleartext. Such usernames and passwords are tricky –but not impossible– to “sniff” with typical 'switched' Ethernets. Even this tiny risk of exposing such important usernames and passwords may not be acceptable in your environment [although in reality keyloggers present a far larger risk].)

(If your configuration requires installing Samba software on your gateway/firewall, be sure Samba listens on the internal/LAN interface but not on the Internet interface. If necessary specify the interfaces parameter in the Samba configuration.)

Operational Overview

Squid<->Browser Exchange

The initial HTTP request is rejected by the proxy server (probably Squid) as not having the needed authentication. The client repeats the request with an authorization header containing host identification.

Once again the request is rejected by the proxy server, but this time with an authorization challenge (a “nonce”). The client repeats the request a third time, this time with the authorization header containing the authorization response (properly encoded nonce proving it knows the password) and the user identification and a repeat of the host identification.

Dansguardian “sniffs” this information so it's aware of the user. Actual authentication validation is then provided by the local proxy server (probably Squid).

The last part of the NTLM handshake is repeated for every connection, so automatic response by the browser is virtually required to eliminate the user having to type the same password over and over.

Squid<->Helper Exchange (Helper Protocol)

What's common in all cases with the “ntlm_auth” helper program is the credentials are checked against Samba. This is true no matter what stdio protocol is specified with --helper-protocol=… or how the credentials are obtained.

The original communication protocol between Squid and any helper auth program was pretty simple: Squid wrote username password and read either OK or ERR. This communication protocol is sometimes called the “stdio protocol” or the “standard I/O protocol”.

The original simple protocol wouldn't work though for multi-exchange auth transactions such as challenge-response like NTLM (or digest). As a result a new protocol was invented, then revised several times. The auth helper program has to exchange information with the main Squid program in some well-defined way …but because of all the options and revisions it isn't always immediately clear what that is. So the protocol for the exchange between the ntlm auth helper program and the Squid main program is explicitly specified by the --helper-protocol=… parameter.

The selected helper protocol should match the Squid auth scheme, which is the very first parameter in the Squid configuration statement auth_param (like the following:).

auth_param {auth-scheme} program ...ntlm_auth --helper-protocol={which-stdio-protocol} ...

Helper protocol options whose names include “client” are for use with other programs and will not work with Squid. Options whose names include “server” are primarily for use with other programs, but may work with Squid anyway in some cases. Of the remainder, options whose names include “basic” match the “auth_param basic ..” scheme, and options whose names include “ntlm” (or “spnego” ?) match the “auth_param ntlm …” scheme.

(You shouldn't see “BH” anywhere in any trace. It stands for “Broken Helper”, indicating something is quite wrong.)

Negotiation

The request for credentials to the Internet Explorer (IE) browser begins with “negotiation”. Supposedly the server (i.e. the proxy, such as Squid), can simply choose between two different forms of negotiation, one of which includes only NTLM and the other of which also includes Kerberos (often called “integrated Windows authentication”).

DansGuardian (but not recent versions of Squid) does not support encrypted Kerberos auth. Even if you correctly configure Squid, Samba, and IE to exchange and validate encrypted Kerberos credentials, DansGuardian will not be able to extract usernames.

This restriction does not apply to the Squid “LDAP” helper, as it handles un-encrypted Kerberos credentials in such a way that the exchange between the Broswer and Squid appears to be BASIC authentication.

But the situation is really more complex, including several other questions: Is Basic considered as one of the options to be negotiated as well as NTLM? Is the LM (Lan Manager) option also included? Is that NTLM or NTLMv2 or both?

Some of these choices can be influenced by server (Squid? Squid “Helper”? even Samba?) configuration, some are influenced by browser version and patch level, some can be influenced by a browser option setting, and influencing some requires changing largely undocumented Registry settings. (And of course the browser options and Registry settings maychange from one version or patch level to the next.) To make things even stranger, an option is often influenced not by just by one of these but rather by some combination of server and client settings.

This document tries to present choices which will work in all cases so none of these details will matter. But despite this best effort, it's still possible that resolving some obscure detailed NTLM issues may require you to scour the web or read Microsoft documentation very closely or trace network traffic or some combination of these.

Fallback Operation

When NTLM credentials are requested of a browser, the browser may do any one of several things:

  • skip the NTLM authentication exchange altogether, instead using some other authentication protocol if the option is available (or not exchanging authentication credentials at all if NTLM is the only choice)
  • automatically fetch credentials from the OS and exchange them    the classic NTLM behavior
  • if this is the first request in a while, automatically fetch credentials from the OS and exchange them; but if this is a follow-on request in quick succession, use a (perhaps different) popup to fetch credentials from the user instead and exchange those
  • always fetch credentials from the user with a (perhaps different) popup and exchange them with no indication they weren't actually fetched from the OS

The first option is what old browsers did when confronted with an NTLM request. The second option is how IE behaved. Many modern browsers do the third option when running under Windows. Modern browsers running on an OS that cannot supply credentials (often a non-Windows platform) often do the fourth option.

To make things even more confusing, the same browser may act differently depending on: i) the exact version and even patch level, ii) configuration parameters (often obscure), or iii) what OS the browser is running under (for example browsers running under Linux can't fetch credentials from the OS because the OS never knew and doesn't remember the login password at all). Some “password plugin extensions” behave very much as though credentials were fetched from the OS. And sometimes a browser with a particular configuration will automatically fetch credentials from the OS and exchange them as Basic (or Digest) credentials rather than NTLM credentials.

Originally it was simple: if credentials were fetched from the OS automatically it was an NTLM exchange, and if a popup appeared it was a Basic exchange. But no more. Now it's not always the case that when credentials are supplied automatically, the browser has been asked for an NTLM exchange. And when the browser uses a popup to ask the user to type credentials, the browser isn't necessarily participating in a Basic exchange but might be participating in an NTLM (or Digest) exchange instead, and the appearance of the popup isn't necessarily the same.

Just because a browser displays a popup requesting the user to manually type credentials does not necessarily mean that the HTTP headers are performing a BASIC credentials exchange.

Squid configurations that may originate both NTLM and BASIC credentials exchanges are often identified by containing both auth_param basic … and auth_param ntlm … statements. FIXME It may be that some combinations of some parameters to some NTLM “helper” programs cause the proxy to emit BASIC credentials exchange requests. If so, this may be another case where both the NTLM and BASIC DansGuardian auth plugins must be enabled.

(To ascertain that BASIC credential exchanges are really being used, you may need to use some debugging technique such as sniffing network packets or running a debug version of DansGuardian or using a tracing local proxy such as "HTTP Analyzer". Usually you cannot tell reliably and accurately just from what appears on the client computer's screen [especially not with IE, which sometimes changes the appearance of a popup for mysterious reasons].)

One way to guarantee correct operation in all cases is to both:

  1. Configure the local proxy (probably Squid) to explicitly request Basic credentials if its previous request for NTLM credentials either was rejected or produced credentials that didn't check out.
  2. Configure DansGuardian to explicitly sniff both NTLM and Basic auth exchanges.
No Auth Popup/Prompt (No Fallback)

One problem that occasionally occurs is: “guest” computers on a network configured for both NTLM and fallback to BASIC auth never show any prompt at all.

It's easy to go down the wrong path, either by assigning different meanings to IE popups that look a little different (something that's notoriously error prone), or by asking “why isn't a BASIC prompt displayed?” When faced with this error, your question should be “why is NTLM auth succeeding?” As the browser's automagic response to the NTLM query is apparently being accepted, the auth sequence is ending without ever falling back to BASIC and displaying a prompt.

What's usually happening is the “guest” computers don't have any users defined, but rather simply login as the local 'Administrator' with the default password. When queried for NTLM credentials, the 'Administrator/***' response from the browser is being accepted by Squid. (Many configurations replace whatever Domain the browser sends with the name of the local Domain, so differing Domain information has no effect.) Thus NTLM auth has succeeded, Squid doesn't send any further queries, and DansGuardian assigns a filter group using the user name 'Administrator'.

If this is happening, try to figure out why your AD has an entry for the user named 'Administrator' with the default password (probably a security risk), and delete it if possible. (An alternative response is to explicitly add 'Administrator' to 'filtergroupslist', understanding that most “guest” computers will be assigned to just this one filter group.)

Detailed Instructions

Often setting up NTLM authentication proceeds easily. But when things go awry, they frequently go really awry. So don't start on this just an hour before production time.

Configuration of NTLM authentication is quite version- and environment-dependent. Pay attention to your own documentation on your own system (especially your Samba), including relevant comments in files like squid.conf.

Shared Details - Samba

Note in general when setting up any Samba server that the 'workgroup=…' in smb.conf specifies what's usually thought of as the domain name, and that the server name is not specified anywhere in smb.conf. The server name (displayed by net server …) is the OS's name of the system (very similar to what the `hostname` command displays), and is fetched from the kernel at runtime by Samba. (The OS domainname -as shown by the`domainname` command- is ignored by Samba and is irrelevant.)

Be sure you have the Samba “root” <user> and <password> to substitute in on the command line of any Samba executable that needs them. You must specify a Samba account, not a *nix account; it's likely that you will not be able to use any *nix account –not even 'root'– anyplace where <user> and <password> are separated by a percent sign. You probably already have the Samba superuser account from the process of installing Samba.

If for some reason Samba does not already have a satisfactory superuser account, you can construct one. Use Samba procedures to create a user (usually just a regular user, but some versions of Samba may have special procedures), except with a) name 'root', b) uidNumber “0” c) sambaSID ”<domainSID>-500” and d) sambaPrimaryGroupSID ”<domainSID>-512”. (The very last short portion of a full SID -typically the part after the last dash- is called the RID [Relative ID]. Many RIDs in the 5xx range have hard-coded meanings to Windows. Note a common documentation error specifies the Domain Admins RID as 200 [decimal] rather than 0×200 [hex], making it seem it's not really 512. Disregard any effects of this documentation error.)

Details Option #1 - Samba-Only

(NT4-style domain, local Samba either Master/PDC or Slave/BDC)

This example presents only the auth-related portions of the configuration. We assume non-authenticated web access already works. We assume Samba is installed and working. And we assume either that all the end user computers are members of the same NT/Samba Domain, or that multi-domain trusts and mappings are already set up.

Furthermore we would like to assume the DansGuardian/Squid server has already joined the NT/Samba Domain. This is generally the case; usually installation of Samba also takes care of making the server a member of its own domain. But double-check the server really is a member of the domain before proceeding, as follows.

Then execute:

{proxy]# net server -U <user>%<password>
Enumerating servers in this domain or workgroup:
 .
...
 .
[proxy]#

If you see the name of the server you're working at in the displayed list, proceed. But if you don't, join the server to the domain first. This might be simply a matter of successfully executing:

[proxy]# net rpc join [PDC|BDC| ] -U <user>%<password>

(The first paramater to the command specifies the “role” your computer will fulfill, Primary Domain Controller, Backup Domain Controller, or neither.)

Use a different variant instead of net rpc … if joining an Active Directory domain that uses Kerberos credentials. Don't forget, these instructions you are reading are specific to Samba-only NT4-style domains; instructions for joining AD are in a different sub-section.

It's fairly likely this simple command won't work, in which case it seems that your installation of Samba was not complete or not completely successful. Revisit your Samba installation process, or keep tinkering. Until your server is a member of your own domain, the NTLM authentication described below definitely won't work and there's no point in proceeding.

There are two NTLM authentication Samba helper programs: one comes in the Squid package; and another one comes in the Samba package. They provide nearly identical functionality, and in some release versions they even have the same name. Use the program from the Samba package (not the one from the Squid package). It's more likely to work even in oddball circumstances, and it's maintained more closely. (To avoid duplication of effort, the Squid developers have decided to pay less attention to their version of the NTLM authentication Sama helper program.)

(The daemon program that implements the service named winbind is named winbindd [that double 'd' at the end isn't just a stutter]. Even this document sometimes gets tripped up and uses one name where the other would be more appropriate. Please disentangle this yourself as necessary as you read.)

Check if winbind is already set up and already running.

If you see lines like the following in your smb.conf it suggests it's likely that winbind is already set up (for another reason unrelated to “auth”).

idmap uid = 10000-20000
# or the same thing with a different spelling:
# winbind uid = 10000-20000
idmap gid = 10000-20000
# or the same thing with a different spelling:
# winbind gid = 10000-20000

If the test below returns a positive result it indicates winbindd is already running.

[proxy]# wbinfo -p
Ping to winbindd succeeded on fd 4
[proxy]#

If winbind is not yet set up, you may need to configure the ID mapping function of Samba. (The same winbind service implements several largely unrelated functions, such as “auth” and “ID mapping”. To get the daemon to run right so “auth” can use it, you need to first get “ID mapping” fully set up. That's why these instructions are needed here, even though they don't at first seem to have much to do with “auth”.)

(The Samba ID mapping configuration is not related to “auth” at all in the case of only a single NT/Samba Domain [except they both rely on the same winbindd]. However in the case where end user computers from other AD Domains might use this DansGuardian/Squid, the ID mapping function of Samba may need to be extended even beyond what's necessary to make Samba store files for the local domain.)

Setting up winbind might involve adding lines similar to idmap uid = … and idmap gid = … and more (in a few cases perhaps much more) to your smb.conf. Details are not included here because they can be quite different with different versions of Samba in different environments. (Currently Samba changes and developments in this area are happening very quickly.) If you have only one NT/Samba Domain, and if the Samba uid/gids are guaranteed to never ever overlap with any local user defined in /etc/passwd or local group defined in /etc/group, you probably don't need any ID mapping at all (although you may nevertheless need to specify a minimal ID mapping configuration, perhaps something like idmap  uid = 16777216-33554431 and idmap gid = 16777216-33554431). But often (including the case where uid's or gid's might sometimes overlap), ID mapping is integral to the Samba file storing functionality. Only if there are multiple NT/Samba domains might more complex ID mapping configurations be necessary.

If winbindd is not yet running, start it (probably either service winbind start or /etc/init.d/winbind start.

Also configure your system so the winbind service automatically starts at reboot. Some systems provide this as an option in a system-wide configuration tool (usually a GUI or a semi-GUI); other systems provide it via the update-rc.d command or the chkconfig command. If your system relies on chkconfig, execute chkconfig --list winbind to see whether the service already starts automatically at reboot, and execute chkconfig winbind on if it doesn't.

Modify your winbindd setup for auth.

First, in smb.conf, make sure it says:

winbind use default domain = yes

This setting says that if any request comes to winbindd without a domain specified (just user password rather than domain\user password), winbindd should simply add the name of the local domain. (Be warned that some directions say [?] that it's necessary to specify just the opposite [winbind use default domain = no] if your DansGuardian/Squid system is serving multiple trusted domains. Possibly confusing is Samba's use of the term “trusted domains” may mean almost the opposite of the Microsoft use of those same words. )

Also in smb.conf check the setting winbind trusted domains only, and try changing it if necessary. It supposedly enables use of Linux (non-Samba) credentials too by stripping off the domain name; this would also seem to be relevant for “local” logins where the domain name is the name of the local workgroup. However the way Samba implements this is greatly influenced by /etc/nsswitch.conf, so it's difficult to even say exactly what it will do (let alone say what the “correct” setting is). Keep in mind that what Samba means by the term “trusted domains” is different than what Microsoft means by the same term.

Next, make sure Squid has read access to the directory “winbindd_privileged” (usually located in /var/cache/samba/ so its full name is /var/cache/samba/winbindd_privileged). Do this by setting the directory's “group” to the group Squid runs as (usually either “squid” or “proxy”, reference 'cache_effective_group' in squid.conf) so it looks something like this:

[proxy]# cd /var/cache/samba
[proxy]# ls -ld winbindd_privileged
drwxr-x---    2 root     squid        4096 Aug 11 12:46 winbindd_privileged
[proxy]# 

(If winbindd starts but won't stay running, check the permissions on this directory. If they allow read access to “others” [drwxr-xr-x]. change them so they don't [drwxr-x---], perhaps with a command like chmod o-rwx winbindd_privileged. This is a common problem due to some incorrect documentation.)

Restart the winbind service to make these changes take effect.

(Wherever <domain> and <user> and <password> are used below, substitute your own domain name and the properly defined credentials of a test user. Avoid using a “real” user to minimize the risk of exposing a user password or messing up their account or polluting usage statistics.)

Check to make sure winbindd can provide authentication. When you execute the test command with valid credentials, you should get output like that shown below, not a bunch of messages including words like “failed” or “error code”. (NTLM authentication with Samba is critically dependent on winbindd, it won't work at all without winbindd.)

[proxy]# wbinfo -a <domain>\\<user>%<password> 
plaintext password authentication succeeded
challenge/response password authentication succeeded
[proxy]#

(Double the backslash as shown so one backslash is left for wbinfo even though the Linux shell strips off the other one.)

This command will fail if you're not running as the right user and the failure does not mean anything is wrong. In order to have completed the previous steps and gotten this far, it's likely you've been running as 'root'. However if you've properly been using a privileged (but non-root) administrative user account, this is the first step where you will encounter problems. You will need to `sudo…` (or `su root`) here.

FIXME What are the minimum requirements for a user to run the above wbinfo command successfully? Being 'root' seems risky and unneccessary. If you know, please fill in the answer in the above paragraph.

Now check that the ntlm_auth helper program can communicate properly with winbindd to provide authentication: (Note in this test we use the plaintext exchange rather than the challenge/response exchange. This is because executing ntlm_auth directly with an encoded response would always fail without an NTLM-aware wrapper program. Fortunately even a plaintext test like this exercises the vast majority of the functionality.)

[proxy]# echo <domain>\\<user> <password> | ntlm_auth --helper-protocol=squid-2.5-basic
[2008/08/31 09:27:21, 3] utils/ntlm_auth.c:check_plaintext_auth(292)
  NT_STATUS_OK: Success (0x0)
OK
[proxy]#

(Double the backslash as shown so one backslash is left for wbinfo even though the Linux shell strips off the other one. The output lines almost certainly won't look exactly like this [there may not even be the same number of lines] and that's okay. All you really care about is that the last line of the output says “OK”. If you see “BH” [Broken Helper] instead, you've mis-typed the –helper-protocol command line argument for this test.)

Add lines like this to squid.conf, then restart it:

# Squid's part of authentication is to request authentication credentials

# The order of these lines is important, 
#  as it determines the sequence of auth methods Squid should try.
# (Most of the parameter values below are just suggestions, 
#  and the path to ntlm_auth may be different on your system.)


# First try NTLM authentication.

# What to use to authenticate
auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
# How many processes for simultaneous service do we start
auth_param ntlm children 30
# How many requests before re-authenticating
auth_param ntlm max_challenge_reuses 100
# How long before re-authenticating
auth_param ntlm max_challenge_lifetime 10 minutes

# If that doesn't work (Linux, non-NTLM aware browsers, local logins, etc.), 
#  next try Basic authentication.
# (Typically with Basic auth the user must type username, password, and domain.)

# What to use to authenticate
auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
# Authentication popup title
auth_param basic realm DansGuardian
# How many processes for simultaneous service do we start
auth_param basic children 5
# For how long are the credentials valid until new auth request
auth_param basic credentialsttl 2 hours
# Win 9x/ME aren't case sensitive, so if we want to use them
auth_param basic casesensitive off


# Create the auth ACL (required scaffolding for use below)
acl authenticated proxy_auth REQUIRED


# Allow everyone to access the web after they have been authenticated
# (By requiring "authenticated", which we defined above to in turn require "proxy_auth",
#  we trigger the entire sequence of requesting authentication credentials.)
# (Place this only after any other access rules you may have: 
#  manager access, Safe_ports, SSL_ports, etc. etc.)
http_access allow authenticated 
# Deny everyone else
http_access deny all

Uncomment lines like these below in dansguardian.conf, then restart it. (The exact path may be different for your installation. Usually the configuration file lines already reflect the correct path so all you need to do is uncomment them.)

authplugin = '/usr/local/etc/dansguardian/authplugins/proxy-basic.conf'
authplugin = '/usr/local/etc/dansguardian/authplugins/proxy-ntlm.conf'
Details Option #2 - Non-Samba Master

FIXME Please add information here.
Complete instructions aren't necessary (although they would be nice); please for starters just supply an actual working configuration.

The general approach is to install an unused slave Samba and identify it to AD as either a BDC or a simple File Server. Make the proxy (Squid?) communicate with this Samba just as though the whole system were Samba. Then this Samba will in turn communicate with the larger AD system using secure protocols. However this information is tentative.

NTLM Manual Entry

Many browsers after automatic NTLM fails will display a two-line popup requesting credentials. But then no matter what you type the credentials will always be refused. This is because NTLM requires three pieces of information: the domain name, the user id, and the password.

But how do you type three pieces of information into a popup with only two lines? In the username box, type both the domain name and the user id. Separate them with a backslash and no spaces. So for example in the two lines you might type cube\janey and ********. (Type the real password. Regardless of what you type, asterisks will be displayed so your password won't be visible to someone looking over your shoulder.)