PK11_Authenticate failed with SEC_ERROR_BAD_PASSWORD (-8177)

Added a USB-token into nssdb path using modutil linking its vendor lib and slot/token name. The same nssdb path has been selected in certificate path in LibreOffice (Tools --> Options --> Security --> Certificate) and its listing the certificate correctly.

Using libnss, we tried to simulate listing of certificates using the below code:

PK11_SetPasswordFunc( GetPasswordFunction ) ; // GetPasswordFunction() returns the password string just like that without any user input.

if (NSS_InitReadWrite("/home/<user>/.pki/nssdb") == SECSuccess)  
{
    printf("\n----------->NSS_Init successful");
    cert_handle = CERT_GetDefaultCertDB();

    if (cert_handle == NULL)
    {
        error = PR_GetError();
        printf("\n----------->Unable to get cert_handle:%s (%d)", PR_ErrorToName(error), (int)error);
    }
    else
    {
        printf("\n----------->Got cert_handle");

        PK11SlotList * slotList = PK11_GetAllTokens( CKM_INVALID_MECHANISM, PR_FALSE, PR_FALSE, NULL ) ;

        if(slotList == NULL) {

            error = PR_GetError();
            printf("\n----------->PK11_GetAllTokens failed !:%s (%d)", PR_ErrorToName(error), (int)error);

        } else {

            PK11SlotInfo * usb_token = NULL;

            for (PK11SlotListElement* slotEle = slotList->head ; slotEle != NULL; slotEle = slotEle->next)
            {
                PK11SlotInfo * pSlot = slotEle->slot ;

                if(pSlot != NULL)
                {
                    printf("\n----------->SlotName(%s) TokenName(%s)", PK11_GetSlotName(pSlot), PK11_GetTokenName(pSlot));

                    if(PK11_IsHW(pSlot) && PK11_IsRemovable(pSlot)){ // select the USB-token in the slots list

                        usb_token = pSlot;
                        break;
                    }
                } else {
                    printf("\n----------->pSlot is empty");
                }
            }// end of for

            if(usb_token != NULL){

                printf("\n----------->Found USB-TOKEN SlotName(%s) TokenName(%s)", PK11_GetSlotName(usb_token), PK11_GetTokenName(usb_token));

                if (PK11_NeedLogin(usb_token)){

                    SECStatus nRet = PK11_Authenticate(usb_token, PR_TRUE, NULL);

                    if(nRet != SECSuccess){
                        error = PR_GetError();
                        printf("\n----------->PK11_Authenticate failed !:%s (%d)", PR_ErrorToName(error), (int)error);
                        printf("\n----------->PORT_GetError() !:(%d)", PORT_GetError());

                        if(PORT_GetError() != SEC_ERROR_IO) {
                            printf("\n----------->NoPassword Exception");
                        } else {
                            printf("\n----------->Some other Exception");
                        }
                    }else {
                        printf("\n----------->PK11_Authenticate successful !");
                    }
                }
            }

            PK11_FreeSlotList(slotList);
        }
    }

    PK11_SetPasswordFunc( NULL ) ;
    PK11_LogoutAll();
    NSS_Shutdown();
}
else
{

    error = PR_GetError();
    printf("\n----------->NSS_Init failed:%s (%d)", PR_ErrorToName(error), (int)error);
}         

We get the following error even though the returned password in GetPasswordFunction() is correct: PK11_Authenticate failed !:SEC_ERROR_BAD_PASSWORD (-8177)

Any help is appreciated! Thanks in advance!

Comments

[sumeshvm20]

Solved this!

While debugging PK11_Authenticate() and related functions in the flow, it was identified that the password returned from password callback function (as per my sample code --> GetPasswordFunction()) was NULL. This was odd as I had already debugged that function and made sure that the password was properly returned.

On checking other functions in libnss, I came to know about PORT_Alloc() which is a memory allocator (secport.h) similar to malloc(). I had used normal C-style string memory allocation. On trying the allocation with PORT_Alloc(), it worked!