Make krb5 wrapper library common so they can be used all over

[Samba/vl.git] / source4 / auth / kerberos / kerberos-porting-to-mit-notes.txt

Copyright Andrew Bartlett <abartlet@samba.org> 2005-2009
Copyright Donald T. Davis <don@mit.edu>        2009

Released under the GPLv3
"Porting Samba4 to MIT-Krb"


         From Idmwiki


IPA v3 will use a version of Samba4 built on top of MIT's Kerberos
implementation, instead of Heimdal's version of Kerberos.

Task list summary for porting changes needed, from Andrew Bartlett:

      * Rewrite or extend the LDAP driver that MIT-KDC will use.
      * MIT KDC changes:  rewrite DAL, add TGS-KBAC, enable PACs,...
      * Full thread-safety for MIT's library code,
      * Many small changes

Task list, without explanations (the list with explanations is in the
later sections of this document):

Porting Samba4 to MIT-krb comprises four main chunks of work:
     1. Rewrite or extend the LDAP driver that MIT-KDC will use:
          a. Our LDAP driver for the KDB needs to know how to do
             Samba4's intricate canonicalization of server names,
             user-names, and realm names.
          b. AD-style aliases for HOST/ service names.
          c. Implicit names for Win2k accounts.
          d. Principal "types":  client / server / krbtgs
          e. Most or all of this code is in 3 source files,
             ~1000 lines in all;
     2. MIT KDC changes
          a. Rewrite the MIT KDC's Data-Abstraction Layer (DAL),
             mostly because he MIT KDC needs to see&   manipulate
             more LDAP detail, on Samba4's behalf;
          b. Add HBAC to the KDC's TGT-issuance, so that Samba4
             can refuse TGTs to kinit, based on time-of-day&
             IP-addr constraints;
          c. turn on MIT-krb 1.7's PAC handling
          d. add bad-password counts, for unified account-lockouts
             across all authT methods (Krb, NTLM, LDAP simple bind,
             etc)
     3. Make sure MIT's library code is more fully thread-safe,
        by replacing all global and static variables with context
        parameters for the library routines.  This may already be
        done.
     4. Many small changes (~15)
          a. some extensions to MIT's libkrb5&   GSSAPI libraries,
             including GSSAPI ticket-forwarding
          b. some refitting in Samba4's use of the MIT libraries;
          c. make sure Samba4's portable socket API works,
             including "packet too large" errors;
          d. MIT's GSSAPI code should support some legacy Samba3
             clients that present incorrectly-calculated checksums;
          e. Samba4 app-server-host holds aUTF-16 PW, plus a
             key bitstring;
          f. in-memory-only credentials cache;
          g. in-memory-only keytab (nice to have);
          h. get OSS NTLM authT library (Likewise Software?);
          i. special Heimdal-specific functions;
          j. principal-manipulation functions;
          k. special check for misconfigured Samba4 hostnames;
          l. improved krb error-messages;
          m. improved krb logging
          n. MS GSSMonger test-suite
          o. testsuite for kpasswd daemon

0. Introduction: This document should be read alongside the Samba4
source code, as follows:

    * For DAL and KDC requirements, please see Samba4's
      source4/kdc/hdb-samba4.c in particular.  This file
      is an implementation against Heimdal's HDB abstraction
      layer, and is the biggest part of the samba-to-krb
      glue layer, so the main part of the port to MIT is
      to replace hdb-samba4 with a similar glue layer
      that's designed for MIT's code.
    * Samba4's PAC requirements are implemeneted in
      source4/kdc/pac-glue.c
    * Both of the above two layers are Heimdal plugins, and
      both get loaded in source4/kdc/kdc.c
    * For GSSAPI requirements, see auth/gensec/gensec_gssapi.c
      (the consumer of GSSAPI in Samba4)
    * For Kerberos library requirements, see
      auth/kerberos/krb5_init_context.c
    * Samba has its own credentials system, wrapping GSS creds,
      just as GSS creds wrap around krb5 creds.  For the
      interaction between Samba4 credential system and GSSAPI
      and Kerberos, see auth/credentials/credentials_krb5.

1. Rewrite or extend the LDAP driver that MIT-KDC will use.

     a. IPA'sLDAP driver for the KDB needs to know how to do
        Samba4's intricate canonicalization of server names,
        user-names, and realm names.
        For hostnames&   usernames, alternate names appear in
        LDAP as extra values in the multivalued "principal name"
        attributes:
          * For a hostname, the alternate names (other than
            the short name, implied from the CN), are stored in
            the servicePrincipalName
          * For a username, the alternate names are stored in
            the userPrincipalName attribute, and can be long
            email-address-like names, such as joe@microsoft.com
            (see "Type 10 names," below).
        GSSAPI layer requirements:  Welcome to the wonderful
        world of canonicalisation.  The MIT Krb5 libs (including
        GSSAPI) do not enable the AS to send kinit a TGT containing
        a different realm-name than what the client asked for,
        even in U/L case differences.  Heimdal has the same problem,
        and this applies to the krb5 layer too, not just GSSAPI.
        There are two kinds of name-canonicalization that can
        occur on Windows:
          * Lower-to-upper case conversion, because Windows domain
            names are usually in upper case;
          * An unrecognizable subsitution of names, such as might
            happen when a user requests a ticket for a NetBIOS domain
            name, but gets back a ticket for the corresponging FQDN.
        As developers, we should test if the AD KDC's name-canonical-
        isation can be turned off with the KDCOption flags in the
        AS-REQ or TGS-REQ;  Windows clients always send the
        Canonicalize flags as KDCOption values.
        Principal Names, long and short names:
        AD's KDC does not canonicalize servicePrincipalNames, except
        for the realm in the KDC reply.  That is, the client gets
        back the principal it asked for, with the realm portion
        'fixed' to uppercase, long form.
        Samba4 does some canonicalization, though Heimdal doesn't
        canonicalize names itself:  For hostnames and usernames,
        Samba4 canonicalizes the requested name only for the LDAP
        principal-lookup, but then Samba4 returns the retrieved LDAP
        record with the request's original, uncanonicalized hostname
        replacing the canonicalized name that actually was found.
        Usernames:  AndrewB says that Samba4 used to return
        the canonicalized username exactly as retrieved from LDAP.
        The reason Samba4 treated usernames differently was that
        the user needs to present his own canonicalized username
        to servers, for ACL-matching.  For hostnames this isn't
        necessary.
        Realm-names:  AD seems to accept a realm's short name
        in krb-requests, at least for AS_REQ operations, but the
        AD KDC always performs realm-canonicalisation, which
        converts the short realm-name to the canonical long form.
        So, this causes pain for current krb client libraries.
        Punchline:  For bug-compatibility, we may need to
        selectively or optionally disable the MIT-KDC's name-
        canonicalization.
        Application-code:
        Name-canonicalisation matters not only for the KDC, but
        also for app-server-code that has to deal with keytabs.
        Further, with credential-caches, canonicalization can
        lead to cache-misses, but then the client just asks for
        new credentials for the variant server-name.  This could
        happen, for example, if the user asks to access the
        server twice, using different variants of the server-name.
        Doubled realm-names:  We also need to handle type 10
        names (NT-ENTERPRISE), which are a full principal name
        in the principal field, unrelated to the realm.  The
        principal field contains both principal&   realm names,
        while the realm field contains a realm name, too, possibly
        different.  For example, an NT-ENTERPRISE principal name
        might look like:  joeblow@microsoft.com@NTDEV.MICROSOFT.COM ,
                          <--principal field-->|<----realm name--->|
        Where joe@microsoft.com is the leading portion, and
        NTDEV.MICROSOFT.COM is the realm.  This is used for the
        'email address-like login-name' feature of AD.
     b.AD-style aliases for HOST/ service names.
        AD keeps a list of service-prefixed aliases for the host's
        principal name.  The AD KDC reads&   parses this list, so
        as to allow the aliased services to share the HOST/ key.
        This means that every ticket-request for a service-alias
        gets a service-ticket encrypted in the HOST/ key.
        For example, this is how HTTP/ and CIFS/ can use the
        HOST/ AD-LDAP entry, without any explicitly CIFS-prefixed
        entry in the host's servicePrincipalName attribute. In the
        app-server host's AD record, the servicePrincipalName says
        only HOST/my.computer@MY.REALM , but the client asks
        for  CIFS/my.omputer@MY.REALM tickets.  So, AD looks in
        LDAP for both name-variants, and finds the HOST/ version,
        In AD's reply, AD replaces the HOST/ prefix with CIFS/ .
        We implement this in hdb-ldb.
        (TBD: Andrew, is this correct?:)
        List of HOST/ aliases:  Samba4 currently uses only a small
        set of  HOST/ aliases: sPNMappings: host=ldap,dns,cifs,http .
        Also, dns's presence in this list is a bug, somehow.
        AD's real list has 53 entries:
        sPNMappings: host=alerter,appmgmt,cisvc,clipsrv,browser,
          dhcp,dnscache,replicator,eventlog,eventsystem,policyagent,
          oakley,dmserver,dns,mcsvc,fax,msiserver,ias,messenger,
          netlogon,netman,netdde,netddedsm,nmagent,plugplay,
          protectedstorage,rasman,rpclocator,rpc,rpcss,remoteaccess,
          rsvp,samss,scardsvr,scesrv,seclogon,scm,dcom,cifs,spooler,
          snmp,schedule,tapisrv,trksvr,trkwks,ups,time,wins,www,
          http,w3svc,iisadmin,msdtc
        Domain members that expect the longer list will break in
        Samba4, as of 6/09.  AB says he'll try to fix this right
        away.  There is another post somewhere (ref lost for the
        moment) that details where in active directory the long
        list  of stored  aliases for HOST/ is.
     c.Implicit names for Win2000 Accounts:  AD keys its
        server-records by CN or by servicePrincipalName, but a
        win2k box's server-entry in LDAP doesn't include the
        servicePrincipalName attribute,  So, win2k server-accounts
        are keyed by the CN attribute instead.  Because AD's LDAP
        doesn't have a servicePrincipalName for win2k servers'
        entries, Samba4 has to have an implicit mapping from
        host/computer.full.name and from host/computer, to the
        computer's CN-keyed entry in the AD LDAP database, so to
        be able to find the win2k server's host name in the KDB.
     d.Principal "types":
        We have modified Heimdal's 'hdb' interface to specify
        the 'class' of Principal being requested.  This allows
        us to correctly behave with the different 'classes' of
        Principal name.  This is necessary because of AD's LDAP
        structure, which uses very different record-structures
        for user-principals, trust principals&   server-principals.
        We currently define 3 classes:
           * client (kinit)
           * server (tgt)
           * krbtgt  the TGS's own ldap record
        Samba4 also now specifies the kerberos principal as an
        explicit parameter to LDB_fetch(), not an in/out value
        on the struct hdb_entry parameter itself.
     e. Most or all of this LDAP driver code is in three source
        files, ~1000 lines in all.  These files are in
        samba4/kdc :
           * hdb-samba4.c  (samba4-to-kdb glue-layer plugin)
           * pac-glue.c    (samba4's pac  glue-layer plugin)
           * kdc.c         (loads the above two plugins).

2. MIT KDC changes

     a.Data-Abstraction Layer (DAL): It would be good to
        rewrite or circumvent the MIT KDC's DAL, mostly because
        the MIT KDC needs to see&   manipulate more LDAP detail,
        on Samba4's behalf.  AB says the MIT DAL may serve well-
        enough, though, mostly as is.  AB says Samba4 will need
        the private pointer part of the KDC plugin API, though,
        or the PAC generation won't work (see sec.2.c, below).
        * MIT's DAL calls lack context parameters (as of 2006),
          so presumably they rely instead on global storage, and
          aren't fully thread-safe.
        * In Novell's pure DAL approach, the DAL only read in the
          principalName as the key, so it had trouble performing
          access-control decisions on things other than the user's
          name (like the addresses).
        * Here's why Samba4 needs more entry detail than the DAL
          provides:  The AS needs to have ACL rules that will allow
          a TGT to a user only when the user logs in from the
          right desktop addresses, and at the right times of day.
          This coarse-granularity access-control could be enforced
          directly by the KDC's LDAP driver, without Samba having
          to see the entry's pertinent authZ attributes.  But,
          there's a notable exception:  a user whose TGT has
          expired, and who wants to change his password, should
          be allowed a restricted-use TGT that gives him access
          to the kpasswd service.  This ACL-logic could be buried
          in the LDAP driver, in the same way as the TGS ACL could
          be enforced down there, but to do so would just be even
          uglier than it was to put the TGS's ACL-logic in the driver.
        * Yet another complaint is that the DAL always pulls an
          entire LDAP entry, non-selectively.  The current DAL
          is OK for Samba4's purposes, because Samba4 only reads,
          and doesn't write, the KDB.  But this all-or-nothing
          retrieval hurts the KDC's performance, and would do so
          even more, if Samba had to use the DAL to change KDB
          entries.
     b.Add HBAC to the KDC's TGT-issuance, so that Samba4 can
        refuse TGTs to kinit, based on time-of-day&   IP-address
        constraints.  AB asks, "Is a DAL the layer we need?"
        Looking at what we need to pass around, AB doesn't think
        the DAL is the right layer; what we really want instead
        is to create an account-authorization abstraction layer
        (e.g., is this account permitted to login to this computer,
        at this time?).  Samba4 ended up doing account-authorization
        inside Heimdal, via a specialized KDC plugin.  For a summary
        description of this plugin API, see Appendix 2.
     c. Turn on MIT-krb 1.7'sPAC handling.
        In addition, I have added a new interface hdb_fetch_ex(),
        which returns a structure including a private data-pointer,
        which may be used by the windc plugin inferface functions.
        The windc plugin provides the hook for the PAC.
     d. Samba4 needsaccess control hooks in the Heimdal&   MIT
        KDCs.  We need to lockout accounts (eg, after 10 failed PW-
        attemps), and perform other controls.  This is standard
        AD behavior, that Samba4 needs to get right, whether
        Heimdal or MIT-krb is doing the ticket work.
        - If PADL doesn't publish their patch for this,
          we'll need to write our own.
        - The windc plugin proivides a function for the main
          access control routines.  A new windc plugin function
          should be added to increment the bad password counter
          on failure.
        - Samba4 doesn't yet handle bad password counts (or good
          password notification), so that a single policy can be
          applied against all means of checking a password (NTLM,
          Kerberos, LDAP Simple Bind, etc).  Novell's original DAL
          did not provide a way to update the PW counts information.
        - Nevertheless, we know that this is very much required in
          AD, because Samba3 + eDirectory goes to great lengths to
          update this information.  This may have been addressed in
          Simo's subsequent IPA-KDC design),
        * AllowedWorkstationNames and Krb5:  Microsoft uses the
          clientAddresses *multiple value* field in the krb5
          protocol (particularly the AS_REQ) to communicate the
          client's netbios name (legacy undotted name,<14 chars)
          AB guesses that this is to support the userWorkstations
          field (in user's AD record).  The idea is to support
          client-address restrictions, as was standard in NT:
          The AD authentication server probably checks the netbios
          address against this userWorkstations value (BTW, the
          NetLogon server does this, too).

3. State Machine safety
when using Kerberos and GSSAPI libraries

    * Samba's client-side&   app-server-side libraries are built
      on a giant state machine, and as such have very different
      requirements to those traditionally expressed for kerberos
      and GSSAPI libraries.
    * Samba requires all of the libraries it uses to be "state
      machine safe" in their use of internal data.  This does not
      necessarily mean "thread safe," and an application could be
      thread safe, but not state machine safe (if it instead used
      thread-local variables).  so, if MIT's libraries were made
      thread-safe only by inserting spinlock() code, then the MIT
      libraries aren't yet "state machine safe."
    * So, what does it mean for a library to be state machine safe?
      This is mostly a question of context, and how the library manages
      whatever internal state machines it has.  If the library uses a
      context variable, passed in by the caller, which contains all
      the information about the current state of the library, then it
      is safe.  An example of this state is the sequence number and
      session keys for an ongoing encrypted session).
    * The other issue affecting state machines is 'blocking' (waiting for a
      read on a network socket).  Samba's non-blocking I/O doesn't like
      waiting for libkrb5 to go away for awhile to talk to the KDC.
    * Samba4 provides a hook 'send_to_kdc', that allows Samba4 to take over the
      IO handling, and run other events in the meantime.  This uses a
      'nested event context' (which presents the challenges that the kerberos
      library might be called again, while still in the send_to_kdc hook).
    * Heimdal has this 'state machine safety' in parts, and we have modified
      Samba4's lorikeet branch to improve this behaviour, when using a new,
      non-standard API to tunnelling a ccache (containing a set of tickets)
      through the gssapi, by temporarily casting the ccache pointer to a
      gss credential pointer.  This new API is Heimdal's samba4-requested
      gss_krb5_import_cred() fcn;  this will have to be rewritten or ported
      in the MIT port.
    * This tunnelling trick replaces an older scheme using the KRB5_CCACHE
      environment variable to get the same job done.  The tunnelling trick
      enables a command-line app-client to run kinit tacitly, before running
      GSSAPI for service-authentication.  The tunnelling trick avoids the
      more usual approach of keeping the ccache pointer in a global variable.
    * [Heimdal uses a per-context variable for the 'krb5_auth_context',
      which controls the ongoing encrypted connection, but does use global
      variables for the ubiquitous krb5_context parameter. (No longer true,
      because the krb5_context global is gone now.)]
    * The modification that has added most to 'state machine safety' of
      GSSAPI is the addition of the gss_krb5_acquire_creds() function.
      This allows the caller to specify a keytab and ccache, for use by
      the GSSAPI code.  Therefore there is no need to use global variables
      to communicate this information about keytab&   ccache.
    * At a more theoretical level (simply counting static and global
      variables) Heimdal is not state machine safe for the GSSAPI layer.
      (But Heimdal is now (6/09) much more nearly free of globals.)
      The Krb5 layer alone is much closer, as far as I can tell, blocking
      excepted. .
    * As an alternate to fixing MIT Kerberos for better safety in this area,
      a new design might be implemented in Samba, where blocking read/write
      is made to the KDC in another (fork()ed) child process, and the results
      passed back to the parent process for use in other non-blocking operations.
    * To deal with blocking, we could have a fork()ed child per context,
      using the 'GSSAPI export context' function to transfer
      the GSSAPI state back into the main code for the wrap()/unwrap() part
      of the operation.  This will still hit issues of static storage (one
      gss_krb5_context per process, and multiple GSSAPI encrypted sessions
      at a time) but these may not matter in practice.
    * This approach has long been controversial in the Samba team.
      An alternate way would be to be implement E_AGAIN in libkrb5:  similar
      to the way to way read() works with incomplete operations.  to do this
      in libkrb5 would be difficult, but valuable.
    * In the short-term, we deal with blocking by taking over the network
      send() and recv() functions, therefore making them 'semi-async'.  This
      doens't apply to DNS yet.These thread-safety context-variables will
      probably present porting problems, during the MIT port.  This will
      probably be most of the work in the port to MIT.
      This may require more thorough thread-safe-ing work on the MIT libraries.

4. Many small changes (~15)

   a. Some extensions to MIT'slibkrb5&   GSSAPI libraries, including
      GSSAPI ticket-forwarding:  This is a general list of the other
      extensions Samba4 has made to / need from the kerberos libraries
      * DCE_STYLE : Microsoft's hard-coded 3-msg Challenge/Response handshake
        emulates DCE's preference for C/R.  Microsoft calls this DCE_STYLE.
        MIT already has this nowadays (6/09).
      * gsskrb5_get_initiator_subkey() (return the exact key that Samba3
        has always asked for.  gsskrb5_get_subkey() might do what we need
        anyway).  This routine is necessary, because in some spots,
        Microsoft uses raw Kerberos keys, outside the Kerberos protocols,
        as a direct input to MD5 and ARCFOUR, without using the make_priv()
        or make_safe() calls, and without GSSAPI wrappings etc.
      * gsskrb5_acquire_creds() (takes keytab and/or ccache as input
        parameters, see keytab and state machine discussion in prev section)
      * The new function to handle the PAC fully
        gsskrb5_extract_authz_data_from_sec_context()
        need to test that MIT's PAC-handling code checks the PAC's signature.
      * gsskrb5_wrap_size (Samba still needs this one, for finding out how
        big the wrapped packet will be, given input length).
   b. Some refitting in Samba4's use of the MIT libraries;
   c. Make sure Samba4'sportable socket API   works:
      * An important detail in the use of libkdc is that we use samba4's
        own socket lib.  This allows the KDC code to be as portable as
        the rest of samba, but more importantly it ensures consistancy
        in the handling of requests, binding to sockets etc.
      * To handle TCP, we use of our socket layer in much the same way as
        we deal with TCP for CIFS.  Tridge created a generic packet handling
        layer for this.
      * For the client, samba4 likewise must take over the socket functions,
        so that our single thread smbd will not lock up talking to itself.
        (We allow processing while waiting for packets in our socket routines).
        send_to_kdc()  presents to its caller the samba-style socket interface,
        but the MIT port will reimplement send_to_kdc(), and this routine will
        use internally the same socket library that MIT-krb uses.
      * The interface we have defined for libkdc allows for packet injection
        into the post-socket layer, with a defined krb5_context and
        kdb5_kdc_configuration structure.  These effectively redirect the
        kerberos warnings, logging and database calls as we require.
      * Samba4 socket-library's current TCP support does not send back
        'too large' error messages if the high bit is set.  This is
        needed for a proposed extension mechanism (SSL-armored kinit,
        by Leif Johansson<leifj@it.su.se>), but is currently unsupported
        in both Heimdal and MIT.
   d. MIT's GSSAPI code should support some legacy Samba3
      clients that presentincorrectly-calculated checksums.
    * Old Clients (samba3 and HPUX clients) use 'selfmade'
      gssapi/krb5 tokens for use in the CIFS session setup.
      These hand-crafted ASN.1 packets don't follow rfc1964
      (GSSAPI) perfectly, so server-side krblib code has to
      be flexible enough to accept these bent tokens.
    * It turns out that Windows' GSSAPI server-side code is
      sloppy about checking some GSSAPI tokens' checksums.
      During initial work to implement an AD client, it was
      easier to make an acceptable solution (acceptable to
      Windows servers) than to correctly implement the
      GSSAPI specification, particularly on top of the
      (inflexible) MIT Kerberos API.  It did not seem
      possible to write a correct, separate GSSAPI
      implementation on top of MIT Kerberos's public
      krb5lib API, and at the time, the effort did not
      need to extend beyond what Windows would require.
    * The upshot is that old Samba3 clients send GSSAPI
      tokens bearing incorrect checksums, which AD's
      GSSAPI library cheerfully accepts (but accepts
      the good checksums, too).  Similarly, Samba4's
      Heimdal krb5lib accepts these incorrect checksums.
      Accordingly, if MIT's krb5lib wants to interoperate
      with the old Samba3 clients, then MIT's library will
      have to do the same.
    * Because these old clients use krb5_mk_req()
      the app-servers get a chksum field depending on the
      encryption type, but that's wrong for GSSAPI (see
      rfc 1964 section 1.1.1). The Checksum type 8003
      should be used in the Authenticator of the AP-REQ!
      That (correct use of the 8003 type) would allow
      the channel bindings, the GCC_C_* req_flags and
      optional delegation tickets to be passed from the
      client to the server.  However windows doesn't seem
      to care whether the checksum is of the wrong type,
      and for CIFS SessionSetups, it seems that the
      req_flags are just set to 0.  This deviant checksum
      can't work for LDAP connections with sign or seal,
      or for any DCERPC connection, because those
      connections do not require the negotiation of
      GSS-Wrap paraemters (signing or sealing of whole
      payloads).  Note:  CIFS has an independent SMB
      signing mechanism, using the Kerberos key.
    * For the code that handles the incorrect&   correct
      checksums, see heimdal/lib/gssapi/krb5/accept_sec_context.c,
      lines 390-450 or so.
    * This bug-compatibility is likely to be controversial
      in the kerberos community, but a similar need for bug-
      compatibility arose around MIT's&   Heimdal's both
      failing to support TGS_SUBKEYs correctly, and there
      are numerous other cases.
      seehttps://lists.anl.gov/pipermail/ietf-krb-wg/2009-May/007630.html
    * So, MIT's krb5lib needs to also support old clients!
   e. Samba4 app-server-host holds aUTF-16 PW, plus a key bitstring;
      See Appendix 1, "Keytab Requirements."
   f.In-memory-only credentials cache   for forwarded tickets
      Samba4 extracts forwarded tickets from the GSSAPI layer,
      and puts them into the memory-based credentials cache.
      We can then use them for proxy work.  This needs to be
      ported, if the MIT library doesn't do it yet.
   g.In-memory-only keytab   (nice to have):
      Heimdal used to offer "in-memory keytabs" for servers that use
      passwords.  These server-side passwords were held in a Samba LDB
      database called secrets.ldb .  The heimdal library would fetch
      the server's password from the ldb file and would construct an
      in-memory keytab struct containing the password, somewhat as if
      the library had read an MIT-style keytab file.  Unfortunately,
      only later, at recv_auth() time, would the Heimdal library convert
      the server-PW into a salted-&-hashed AES key, by hashing 10,000
      times with SHA-1.  Naturally, this is really too slow for recv_auth(),
      which runs when an app-server authenticates a client's app-service-
      request.  So, nowadays, this password-based in-memory keytab  is
      falling into disuse.
   h. Get OSSNTLM   authT library: AB says Likewise software
      probably will give us their freeware "NTLM for MIT-krb"
      implementation.
   i. Special Heimdal-specific functions;  These functions didn't
      exist in the MIT code, years ago, when Samba started.  AB
      will try to build a final list of these functions:
      * krb5_free_keyblock_contents()
      *
   j.Principal-manipulation functions:  Samba makes extensive
      use of the principal manipulation functions in Heimdal,
      including the known structure behind krb_principal and
      krb5_realm (a char *).  For example,
      * krb5_parse_name_flags(smb_krb5_context->krb5_context, name,
                              KRB5_PRINCIPAL_PARSE_REQUIRE_REALM,&principal);
      * krb5_unparse_name_flags(smb_krb5_context->krb5_context,    principal,
                              KRB5_PRINCIPAL_UNPARSE_NO_REALM,&new_princ);
      * krb5_principal_get_realm()
      * krb5_principal_set_realm()
      These are needed for juggling the AD variant-structures
      for server names.
   k. SpecialShort name rules   check for misconfigured Samba4
      hostnames;  Samba is highly likely to be misconfigured, in
      many weird and interesting ways.  So, we have a patch for
      Heimdal that avoids DNS lookups on names without a "." in
      them.  This should avoid some delay and root server load.
      (This errors need to be caught in MIT's library.)
   l.Improved krb error-messages;
      krb5_get_error_string():  This Heimdal-specific function
      does a lot to reduce the 'administrator pain' level, by
      providing specific, English text-string error messages
      instead of just error code translations.  (This isn't
      necessary for the port, but it's more useful than MIT's
      default err-handling;  Make sure this works for MIT-krb)
   m.Improved Kerberos logging support:
      krb5_log_facility(): Samba4 now uses this Heimdal function,
      which allows us to redirect the warnings and status from
      the KDC (and client/server Kerberos code) to Samba's DEBUG()
      system.  Samba uses this logging routine optionally in the
      main code, but it's required for KDC errors.
   n. MSGSSMonger   test-suite:  Microsoft has released a krb-specific
      testsuite called gssmonger, which tests interoperability.  We
      should compile it against lorikeet-heimdal&   MIT and see if we
      can build a 'Samba4' server for it.  GSSMonger wasn't intended
      to be Windows-specific.
   o.Testsuite for kpasswd daemon:  I have a partial kpasswd server
      which needs finishing, and a Samba4 needs a client testsuite
      written, either via the krb5 API or directly against GENSEC and
      the ASN.1 routines.  Samba4 likes to test failure-modes, not
      just successful behavior.  Currently Samba4's kpasswd only works
      for Heimdal, not MIT clients.  This may be due to call-ordering
      constraints.


Appendix 1: Keytab Requirements

    Traditional 'MIT' keytab operation is very different from AD's
    account-handling for application-servers:
    a. Host PWs vs service-keys:
       * Traditional 'MIT' behaviour is for the app-server to use a keytab
         containing several named random-bitstring service-keys, created
         by the KDC.  An MIT-style keytab holds a different service-key
         for every kerberized application-service that the server offers
         to clients.  Heimdal also implements this behaviour.  MIT's model
         doesn't use AD's UTF-16 'service password', and no salting is
         necessary for service-keys, because each service-key is random
         enough to withstand an exhaustive key-search attack.
       * In the Windows model, the server key's construction is very
         different:  The app-server itself, not the KDC, generates a
         random UTF-16 pseudo-textual password, and sends this password
         to the KDC using SAMR, a DCE-RPC "domain-joining" protocol (but
         for windows 7, see below).  Then, the KDC shares this server-
         password with every application service on the whole machine.
       * Only when the app-server uses kerberos does the password get
         salted by the member server (ie, an AD server-host).  (That
         is, no salt information appears to be conveyed from the AD KDC
         to the member server, and the member server must use the rules
         described in Luke's mail, in Appendix 3, below).  The salted-
         and-hashed version of the server-host's PW gets stored in the
         server-host's keytab.
       * Samba file-servers can have many server-names simultaneously
         (kind of like web servers' software-virtual-hosting), but since
         these servers are running in AD, these names can be set up to
         all share the same secret key.  In AD, co-located server names
         almost always share a secret key like this.  In samba3, this
         key-sharing was optional, so some samba3 hosts' keytabs did
         hold multiple keys.  Samba4 abandons this traditional "old MIT"
         style of keytab, and only supports one key per keytab, and
         multiple server-names can use that keytab key in common.  In
         dealing with this model, Samba4 uses both the traditional file
         keytab and an in-MEMORY keytabs.
       * Pre-Windows7 AD and samba3/4 both use SAMR, an older protocol,
         to jumpstart the member server's PW-sharing with AD (the "windows
         domain-join process").  This PW-sharing transfers only the PW's
         UTF-16 text, without any salting or hashing, so that non-krb
         security mechanisms can use the same utf-16 text PW.  For
         Windows 7, this domain-joining uses LDAP for PW-setting.
    b. Flexible server-naming
       * The other big difference between AD's keytabs and MIT's is that
         Windows offers a lot more flexibility about service-principals'
         names. When the kerberos server-side library receives Windows-style tickets
         from an app-client, MIT's krb library (or GSSAPI) must accommodate
         Windows' flexibility about case-sensitivity and canonicalization.
         This means that an incoming application-request to a member server
         may use a wide variety of service-principal names.  These include:
               machine$@REALM      (samba clients)
           HOST/foo.bar@realm      (win2k clients)
           cifs/foo.bar@realm      (winxp clients)
               HOST/foo@realm      (win2k clients, using netbios)
               cifs/foo@realm      (winxp clients, using netbios),
         as well as all upper/lower-case variations on the above.
    c. Keytabs&   Name-canonicalization
       * Heimdal's GSSAPI expects to to be called with a principal-name&   a keytab,
         possibly containing multiple principals' different keys.  However, AD has
         a different problem to solve, which is that the client may know the member-
         server by a non-canonicalized principal name, yet AD knows the keytab
         contains exactly one key, indexed by the canonical name.  So, GSSAPI is
         unprepared to canonicalize the server-name that the cliet requested, and
         is also overprepared to do an unnecessary search through the keytab by
         principal-name.  So Samba's server-side GSSAPI calls have to "game" the
         GSSAPI, by supplying the server's known canonical name, with the one-key
         keytab. This doesn't really affect IPA's port of Samba4 to MIT-krb.
       * Because the number of U/L case combinations got 'too hard' to put into
         a keytab in the traditional way (with the client to specify the name),
         we either pre-compute the keys into a traditional keytab or make an
         in-MEMORY keytab at run time.  In both cases we specifiy the principal
         name to GSSAPI, which avoids the need to store duplicate principals.
       * We use a 'private' keytab in our private dir, referenced from the
         secrets.ldb by default.

Appendix 2: KDC Plugin for Account-Authorization

Here is how Samba4 ended up doing account-authorization in
Heimdal, via a specialized KDC plugin.  This plugin helps
bridge an important gap:  The user's AD record is much richer
than the Heimdal HDB format allows, so we do AD-specific
access-control checks in the plugin's AD-specific layer,
not in the DB-agnostic KDC server:
    * We created a separate KDC plugin, with this API:
      typedef struct
           hdb_entry_ex { void     *ctx;
                          hdb_entry entry;
                          void    (*free_entry)(krb5_context, struct hdb_entry_ex *);
           } hdb_entry_ex;
      The void *ctx is a "private pointer," provided by the
      'get' method's hdb_entry_ex retval.  The APIs below use
      the void *ctx so as to find additional information about
      the user, not contained in the hdb_entry structure.
      Both the provider and the APIs below understand how to
      cast the private void *ctx pointer.
      typedef krb5_error_code
            (*krb5plugin_windc_pac_generate)(void * krb5_context,
                                          struct hdb_entry_ex *,
                                                    krb5_pac*);
      typedef krb5_error_code
            (*krb5plugin_windc_pac_verify)(void *  krb5_context,
                                        const   krb5_principal,
                                        struct  hdb_entry_ex *,
                                        struct  hdb_entry_ex *,
                                                krb5_pac *);
      typedef krb5_error_code
            (*krb5plugin_windc_client_access)(void * krb5_context,
                                              struct hdb_entry_ex *,
                                                     KDC_REQ *,
                                                     krb5_data *);
      The krb5_data* here is critical, so that samba's KDC can return
      the right NTSTATUS code in the 'error string' returned to the
      client.  Otherwise, the windows client won't get the right error
      message to the user (such as 'password expired' etc).  The pure
      Kerberos error is not enough)
      typedef struct
              krb5plugin_windc_ftable { int                            minor_version;
                                        krb5_error_code                (*init)(krb5_context, void **);
                                        void                           (*fini)(void *);
                                        krb5plugin_windc_pac_generate   pac_generate;
                                        krb5plugin_windc_pac_verify     pac_verify;
                                        krb5plugin_windc_client_access  client_access;
              } krb5plugin_windc_ftable;
      This API has some Heimdal-specific stuff, that'll
      have to change when we port this KDC plugin to MIT krb.
    * 1st callback (pac_generate)  creates an initial PAC from the user's AD record.
    * 2nd callback (pac_verify)    checks that a PAC is correctly signed,
                                   adds additional groups (for cross-realm tickets)
                                   and re-signs with the key of the target kerberos
                                   service's account
    * 3rd callback (client_access) performs additional access checks, such as
                                   allowedWorkstations and account expiry.
    * For example, to register this plugin, use the kdc's standard
      plugin-system at Samba4's initialisation:
         /* first, setup the table of callback pointers */
        /* Registar WinDC hooks */
         ret = krb5_plugin_register(krb5_context, PLUGIN_TYPE_DATA,
                                    "windc",&windc_plugin_table);
         /* once registered, the KDC will invoke the callbacks */
         /* while preparing each new ticket (TGT or app-tkt)   */
    * An alternative way to register the plugin is with a
      config-file that names a DSO (Dynamically Shared Object).

Appendix 3: Samba4 stuff that doesn't need to get ported.

Heimdal oddities
* Heimdal is built such that it should be able to serve multiple realms
    at the same time.  This isn't relevant for Samba's use, but it shows
    up in a lot of generalisations throughout the code.
* Samba4's code originally tried internally to make it possible to use
    Heimdal's multi-realms-per-KDC ability, but this was ill-conceived,
    and AB has recently (6/09) ripped the last of that multi-realms
    stuff out of samba4.  AB says that in AD, it's not really possible
    to make this work;  several AD components structurally assume that
    there's one realm per KDC.  However, we do use this to support
    canonicalization of realm-names:  case variations, plus long-vs-short
    variants of realm-names.  No MIT porting task here, as long as MIT kdc
    doesn't refuse to do some LDAP lookups (eg, alias' realm-name looks
    wrong).
* Heimdal supports multiple passwords on a client account:  Samba4
    seems to call hdb_next_enctype2key() in the pre-authentication
    routines, to allow multiple passwords per account in krb5.
    (I think this was intended to allow multiple salts).  AD doesn't
    support this, so the MIT port shouldn't bother with this.
Not needed anymore, because MIT's code now handles PACs fully:
* gss_krb5_copy_service_keyblock() (get the key used to actually
    encrypt the ticket to the server, because the same key is used for
    the PAC validation).
* gsskrb5_extract_authtime_from_sec_context (get authtime from
    kerberos ticket)
* gsskrb5_extract_authz_data_from_sec_context (get authdata from
    ticket, ie the PAC.  Must unwrap the data if in an AD-IFRELEVANT)]
Authz data extraction
* We use krb5_ticket_get_authorization_data_type(), and expect
    it to return the correct authz data, even if wrapped in an
    AD-IFRELEVANT container.  This doesn't need to be ported to MIT.
    This should be obsoleted by MIT's new PAC code.
libkdc
* Samba4 needs to be built as a single binary (design requirement),
    and this should include the KDC.  Samba also (and perhaps more
    importantly) needs to control the configuration environment of
    the KDC.
* But, libkdc doesn't matter for IPA; Samba invokes the Heimdal kdc
    as a library call, but this is just a convenience, and the MIT
    port can do otherwise w/o trouble.)
Returned Salt for PreAuthentication
    When the AD-KDC replies to pre-authentication, it returns the
    salt, which may be in the form of a principalName that is in no
    way connected with the current names.  (ie, even if the
    userPrincipalName and samAccountName are renamed, the old salt
    is returned).
    This is the kerberos standard salt, kept in the 'Key'.  The
    AD generation rules are found in a Mail from Luke Howard dated
    10 Nov 2004.  The MIT glue layer doesn't really need to care about
    these salt-handling details;  the samba4 code&   the LDAP backend
    will conspire to make sure that MIT's KDC gets correct salts.
    >
    >    From: Luke Howard<lukeh@padl.com>
    >    Organization: PADL Software Pty Ltd
    >    To: lukeh@padl.com
    >    Date: Wed, 10 Nov 2004 13:31:21 +1100
    >    Cc: huaraz@moeller.plus.com, samba-technical@lists.samba.org
    >    Subject: Re: Samba-3.0.7-1.3E Active Directory Issues
    >    -------
    >
    >    Did some more testing, it appears the behaviour has another
    >    explanation. It appears that the standard Kerberos password salt
    >    algorithm is applied in Windows 2003, just that the source principal
    >    name is different.
    >
    >    Here is what I've been able to deduce from creating a bunch of
    >    different accounts:
    >    [SAM name in this mail means the AD attribute samAccountName .
    >     E.g., jbob for a user and jbcomputer$ for a computer.]
    >
    >    [UPN is the AD userPrincipalName attribute.  For example, jbob@mydomain.com]
    >    Type of account                        Principal for Salting
    >    ========================================================================
    >    Computer Account                host/<SAM-Name-Without-$>.realm@REALM
    >    User Account Without UPN<SAM-Name>@REALM
    >    User Account With UPN<LHS-Of-UPN>@REALM
    >
    >    Note that if the computer account's SAM account name does not include
    >    the trailing '$', then the entire SAM account name is used as input to
    >    the salting principal. Setting a UPN for a computer account has no
    >    effect.
    >
    >    It seems to me odd that the RHS of the UPN is not used in the salting
    >    principal. For example, a user with UPN foo@mydomain.com in the realm
    >    MYREALM.COM would have a salt of MYREALM.COMfoo. Perhaps this is to
    >    allow a user's UPN suffix to be changed without changing the salt. And
    >    perhaps using the UPN for salting signifies a move away SAM names and
    >    their associated constraints.
    >
    >    For more information on how UPNs relate to the Kerberos protocol,
    >    see:
    >
    >    http://www.ietf.org/proceedings/01dec/I-D/draft-ietf-krb-wg-kerberos-referrals-02.txt
    >
    >    -- Luke