PKINIT to Windows AD fails about half the time

["Douglas E. Engert" <deengert@anl.gov>]

While testing the Heimdal-20050405 PKINIT client code with smart
cards and Windows AD, about half the time the AD would return
a KRB_ERROR with error code 60, (generic error) and no
e-text or e-data.

looking closer the difference appears to be that Win2K AD is
expecting the nonce to be a positive int32 or it can't parse the
asn1.

A temporary fix to init_creds_pw.c:

--- ./lib/krb5/,init_creds_pw.c	Wed Feb  2 01:30:25 2005
+++ ./lib/krb5/init_creds_pw.c	Wed Apr 20 13:57:00 2005
@@ -1199,7 +1207,7 @@

      /* Set a new nonce. */
      krb5_generate_random_block (&ctx->nonce, sizeof(ctx->nonce));
-    ctx->nonce &= 0xffffffff;
+    ctx->nonce &= 0x7fffffff; /* shot in dark that win2k wants positive */
      ctx->as_req.req_body.nonce = ctx->nonce;
  #if 0
      krb5_generate_random_block (&ctx->pk_nonce, sizeof(ctx->pk_nonce));


It looks like if the top bit is on, the AS_REQ is one byte larger
then if it is off. I suspect this is asn1 adding a zero byte.

Has this been seen before? There is some code to have a different
pk_nonce, but it is #if'ed out.

Or is this a ans1 encoding problem on the client side.

-- 

  Douglas E. Engert  <DEEngert@anl.gov>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois  60439
  (630) 252-5444