SBL-SEC-00102: Error loading the security adapter library

Applies to:

Siebel System Software - Version 8.0.0.1 [20408] to 8.2.2.2 SIA[23016] [Release V8]
Information in this document applies to any platform.

Product Release: V8 (Enterprise)

Version: 8.0 [20405]

Database: Oracle 10.1.0.4

Application Server OS: Red Hat Linux 4.0

Database Server OS: Sun Solaris 9



This document was previously published as Siebel SR 38-3467659813.

Symptoms

Hi, we're trying to setup a PoC of Siebel 8.0 on Linux Redhat 4.0 and
had encountered issue while trying to config LDAP authentication and an
error had ocurred in the OM log file. IBM LDAP 6.0 Client is installed
and is using OID 10.1.2 as the LDAP Server.

We have repreatedly verified LDAPparameters many times and believed we have setup the right parmeters.

We got the following error in the log file:

SBL-SEC-00102 An error occurred loading the security adapter library. Please contact your system administrator for assistance.

SBL-DAT-00562 An error occurred loading the security adapter library. Please contact your system administrator for assistance


Btw, logfile and siebns are attached for your reference..

Cause

Change Request 12-1LDF7SJ

Solution

Message 1

Please assign this SR as soon as possible. This is becoming critical for the customer .

Many Thanks

Message 2

For the benefit of other readers

The customer was implementing LDAP authentication with Siebel 8.0 on Red Hat Linux 4.0 with Oracle Internet Directory.


After configuring the LDAP Adapter and restarting the Siebel Server the
Object Manager startup would fail with the following errors:


GenericLog GenericError 1 0000000646fb2542:0 2007-09-27 20:38:39
(secmgr.cpp (2407) err=4587622 sys=0) SBL-SEC-00102: An error occurred
loading the security adapter library. Please contact your system
administrator for assistance.
ObjMgrSessionLog Error 1
0000000646fb2542:0 2007-09-27 20:38:39 (physmod.cpp (9244))
SBL-DAT-00562: An error occurred loading the security adapter library.
Please contact your system administrator for assistance.

ObjMgrSessionLog Error 1 0000000646fb2542:0 2007-09-27 20:38:39
(model.cpp (5867)) SBL-DAT-00562: An error occurred loading the security
adapter library. Please contact your system administrator for
assistance.
ObjMgrCTLog Error 1 0000000646fb2542:0 2007-09-27 20:38:39 (ctxtmgr.cpp (4492)) SBL-SVC-00208: Please login first.


The customer confirmed that the IBM LDAP Client 6.0 was already
installed. After further investigation it was found that the LDAP Client
libraries needed to be added to the LD_LIBRARY_PATH set in
/siebsrvr/siebenv.sh. siebenv.sh was modified as follows:

[....]

Message 3

[....]

if [ a${LD_LIBRARY_PATH} = ${LD_LIBRARY_PATH}a ] then

LD_LIBRARY_PATH=${SIEBEL_ROOT}/lib:${SIEBEL_ROOT}/lib/odbc/merant:/opt/ibm/ldap/V6.0/lib:${MWHOME}/lib:${SQLANY}/lib:/usr/lib

else
LD_LIBRARY_PATH=${SIEBEL_ROOT}/lib:${SIEBEL_ROOT}/lib/odbc/merant:/opt/ibm/ldap/V6.0/lib:${MWHOME}/lib:${SQLANY}/lib:/usr/lib:${LD_LIBRARY_PATH}
Fi

After making this change LDAP Authentication worked correctly. Change Request 12-1LDF7SJ was logged for a documentation defect.

Applies to:

Siebel CRM - Version 7.5.3.4 [16180] and later
Information in this document applies to any platform.

*** Checked for relevance on 29-JAN-2013 ***

Symptoms

Customer is trying to implement the LDAP security adapter
(LDAPSecAdpt) for use with the Siebel application.  The application
object manager is failing to bring up a login screen and the following
error is appearing in the log file:

SBL-SEC-00102: Error loading the security adapter library. Please contact your system administrator.

The desired behavior is that the security adapter library should load successfully and a login page be presented to the user.

Cause

This behavior occurred because the server had to be physically
rebooted after the installation of the IBM LDAP Client and/or GSKit
software.

Solution

After some investigation, we determined that it is necessary to
physically reboot the server after the installation of the IBM LDAP
Client or GSKit software. Once this was done, everything worked
correctly.

This requirement is not documented in the Siebel Bookshelf documentation.


Document Enhancement Request Bug:10575898/ 12-1UX6R4P to update the Security Guide with this information.

Applies to:

Siebel Financial Services CRM - Version 8.0.0.1 [20408] to 8.1.1.2[21215] [Release V8]
Information in this document applies to any platform.

***Checked for relevance on 03-Jan-2011***

***Checked for relevance on 23-Nov-2012***

Symptoms

We are trying to implement a Custom Security Adapter to perform
application login authentication. We have downloaded the sample SDK from
support web, compiled and deployed to our testing environment. We have
also configured one of the AOM to test out the compiled security
adapter. Unfortunately, we are unable to load the application login
page. Based on the attached log file, the error message is as below:

"SBL-SEC-00102:
An error occurred loading the security adapter library. Please contact
your system administrator for assistance."

Cause

Incorrect setup/configuration on the custom security adapter named
subsystem and configuration file or security adapter library compilation
issue.

Solution

Verify the following is correct.


- Parameter setting in custom security adapter named subsystem


- Setting in the configuration file reference in ConfigFileName parameter in custom security adapter named subsystem


- LIBPATH environment variable setting

Setup the sdk custom
adapter authentication and configuration in-house and successfully
reproduce the issue when using the same compiler (g++) and makefile as
customer.


Further research on these and found the issue on the sdk compilation. The makefile from customer is like this.

libsampsdk.so: secsamp.o secutil.o
g++ secutil.o secsamp.o -o libsampsdk.so

secsamp.o: secsamp.cpp secutil.h ssecurity.h
g++ -c secsamp.cpp

secutil.o: secutil.cpp secutil.h ssecurity.h
g++ -c secutil.cpp

Note
that after compiling the secsamp.cpp and secutil.cpp with g++ -c swtich
and option, it will create the .o file. Once this dependencies
(secsamp.o and secutil.o) for the shared object secsamp.so have been
created, there is need to invoke the linker using additional switch and
option to create a dynamic library. So modify the makefile as below and
recompile the sample code.

libsampsdk.so: secsamp.o secutil.o
g++ -fPIC -shared secutil.o secsamp.o -o libsampsdk.so

secsamp.o: secsamp.cpp secutil.h ssecurity.h
g++ -c secsamp.cpp

secutil.o: secutil.cpp secutil.h ssecurity.h
g++ -c secutil.cpp

Using this newly compiled library file, object manager and application succesfully loaded and login works fine.

pplies to:

Siebel Communications Service - Version 8.0.0.2 [20412] to 8.1.1.8 SIA [23012] [Release V8]
Information in this document applies to any platform.

Symptoms

Siebel version 8.0 application does not start when the LDAP security
adaptor is configured at enterprise level, although the mandatory IBM
LDAP client version 6.0 has been installed. The following error messages
are logged as a result,


a) SBL-SEC-00102: An error occurred loading the security adapter
library. Please contact your system administrator for assistance.
b)
SBL-DAT-00562: An error occurred loading the security adapter library.
Please contact your system administrator for assistance.

Cause

The Siebel security adaptor library aka. libsscfldap.so could not be
loaded because of the installation and configuration (LD_LIBRARY_PATH)
of the 64bit libraries that are part of the IBM LDAP client software.

Solution

Ensure that the LD_LIBRARY_PATH in the siebenv.sh shell script only
references the 32-bit libraries of the IBM LDAP Client version 6.0
because this ensures that the Siebel modules, which are based on 32-bit
processing - do not conflict with the 64-bit library versions of the IBM
LDAP Client software - as documented in Siebel
Bookshelf version 8.0 > Siebel Security Guide > Security Adapter
Authentication > Installing LDAP Client Software > Configuring the
siebenv.csh and siebenv.sh Scripts for the LDAP Client - where it
states, "Siebel supports only the IBM 32-bit LDAP Client so you can use
only the 32-bit LDAP Client libraries with your Siebel applications."

Please follow the following steps, also outlined in the Siebel Bookshelf reference,
1) Navigate to the Siebel server root installation directory
2) Open a text editor on your UNIX system e.g. vi
3)
For both the shell script files; SIEBENV.SH and SIEBENV.CSH edit them
and ensure that ONLY the location to the IBM LDAP Client 32bit libraries
are referenced.

NOTE: The 64-bit libraries can be explicitly identified from the directory path naming convention - that is,
32-bit  =  /opt/IBM/ldap/V6.0/lib
64-bit  =  /optIBM/ldap/V6.0/lib64

Applies to:

Siebel CRM - Version 8.0.0.1 SIA [20408] and later
Information in this document applies to any platform.

Symptoms

Siebel Login Page fails to display



ERRORS


1) The server you are trying to access is either busy or experiencing
difficulties. Please close the Web browser, open a new browser window,
and try logging in again.[08:22:46]

2) SBL-SEC-00102: An error occurred loading the security adapter
library. Please contact your system administrator for assistance.

Cause

The IBM LDAP Client installation on UNIX defaults to /opt/ibm/ldap/V6.0.


The environment variables in the SIEBENV.SH referenced the wrong directory for the IBM LDAP libraries - namely /opt/ibm/ldap/v6.0/lib as opposed to /opt/ibm/ldap/V6.0/lib.


Because the resolution of filename paths on UNIX systems is case
sensitive the difference between v6.0 and V6.0 caused the error.


Bug 14651929
has been logged requesting that the Security Guide be updated
accordingly under Security Adapter Authentication > Process of
Installing and Configuring LDAP Client Software > Configuring the
siebenv.csh and siebenv.sh Scripts for the LDAP Client.

Solution

Modify the SIEBENV.SH to reference /V6.0/lib as opposed to /v6.0/lib

Applies to:

Siebel CRM Call Center - Version: 8.1.1.5 [21229] and later   [Release: V8 and later ]
Information in this document applies to any platform.

Symptoms

=== ODM Issue Clarification ===

This is in relation to a 8.1.1.5 siebel enterprise environment deployed on a RHEL 5.5 linux platform (64-bit).





When accessing the EAIObjMgr_enu component fails when using an external
Java based command line utility, which is accessing the EAI component
via EAI_enu SWSE web server.



Looking at the EAIObjMgr component we can see mainly authentication errors like the following error messages:

"An error occurred loading the security adapter library. Please contact
your system administrator for assistance.(SBL-DAT-00562)" and before
this error ".....err=4587622....SBL-SEC-00102..."

Cause

=== ODM Cause Determination ===



Executed: strace against start_server and EAI OM PID. There was no evidence that <custom adapter>.so lib was loaded.



Executed: file <custom adapter>.so

This showed: ELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped



Executed: ldd -v <custom adpater>.so

This showed:

>>>

      libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b536dd51000)

      libm.so.6 => /lib64/libm.so.6 (0x00002b536e051000)

      libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b536e2d4000)

      libc.so.6 => /lib64/libc.so.6 (0x00002b536e4e3000)

      /lib64/ld-linux-x86-64.so.2 (0x0000003d04c00000)



      Version information:

      ./cdetsnopassword.so:

              libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6

...

<<<

=== ODM Cause Justification ===



Looking at the file and ldd output, we can see that 64-bit libraries are used:



Siebel product is only 32-bit and will only work with 32-bit libraries.

Solution

=== ODM Solution / Action Plan ===



Compile the custom security adapter as per SDK documented compile commands.

Also ensure that modifications are implemented as per published article:
1062497.1 - Changes to Siebel Custom Security Adapter SDK in Siebel
8.1.1

Also, if on a 64-bit operation system,  ensure to modify the compiler
instructions to compile in 32-bit mode. There could be different ways
this can be done pending on the OS vendor. For any clarification please
consult with the OS vendor support.



In this specific case the following compiler instructions helped to resolve the issue.

g++ -m32 -c -fPIC secsamp.cpp

g++ -m32 -c -fPIC secutil.cpp

g++ -m32 -fPIC -shared secutil.o secsamp.o -o <custom security adapter>.so

Please note, Oracle  is providing the SDK and compiler instructions solely as a convenience to our customers and

Oracle  makes no warranties or representations of any kind regarding the

results customers may achieve by using the SDK. Siebel is not responsible

for any failure of any security adapter written using the SDK.



Check with file and ldd command (library dependency check on Linux) to ensure no 64-bit library is being used.

For example in this case the output after compiling in 32-bit mode confirmed that there was no 64-bit library involved.

$ ldd <custom security adapter>.so

linux-gate.so.1 =>  (0xffffe000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xf7e82000)

libm.so.6 => /lib/libm.so.6 (0xf7e5b000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xf7e4f000)

libc.so.6 => /lib/libc.so.6 (0xf7d09000)

/lib/ld-linux.so.2 (0x00473000)



Stop the siebel application server service.

Replaced this on the siebsrvr/lib

Start the siebel application server service



Test the siebel application authentication process.

SBL-SCB-00015: The component is down or not available

Applies to:

Siebel System Software - Version: 7.0.4 [14068] and later   [Release: V7 and later ]

Generic Windows

Area(s):System Administration

Release(s):V7 (Enterprise), V7 (Professional)

Database(s):All Supported Databases

App Server OS(s):HP-UX, Solaris, AIX

Latest release tested against:V7 (Enterprise)

Keywords:Solaris, accept, failed, shutdown, signal, SIGPIPE, core, SBL-SMI-00006, hang, logout, users



This document was previously published as Siebel Alert 1206.

Description

Process
signals are used for the purpose of interrupting normal process
execution and informing it about a certain error condition that needs
some recovery activity by a process or by a certain thread of a process.

These signals could be generated by the operating system, or could be invoked via C/C++ code.

Examples:

• A
  custom C++ application (library) is invoked from a Call Center Object
  Manager user or thread via the Siebel eScript SELib mechanism. The
  library connects to a process or files via named pipe mechanism. A
  networking problem or timeout occurs and the communication pipe is
  broken unexpectedly. The operating system issues a SIGPIPE signal to the
  process to inform it about the broken connection.

• A
  custom C++ application (library) is invoked from a Call Center Object
  Manager user or thread via the Siebel eScript SELib mechanism. The C++
  application contains code that sets a specified time it wants to perform
  an operation or wait on a particular resource. It uses a SIGALRM
  signal/handler routine to achieve this. After the time period is elapsed
  a SIGALRM process is triggered.

In
both of these cases, the signals are intended to inform the thread
which was performing a particular request to stop (interrupted) and then
to execute a particular signal handler routine.

However,
these signals could also intercepted by the listener thread (thread# 1)
which is responsible for the startup and shutdown of the entire
process. When such a signal is received, the process may shutdown
resulting in all the users who are logged into this process to have to
re-login.

Bug 10501186 has been logged to address this product defect.

Likelihood of Occurrence

You
may encounter the behavior in this Alert if you are running the Siebel
Business Application Server versions 7.7.x or 7.8.x on any
UNIX platform.

Possible Symptoms

Below are different symptoms that may be reported if you are encountering this behavior:

• The browser displays the error message for new users trying to login:

SBL-SWP-00121:
The server you are trying to access is either busy or experiencing
difficulties. Please close the Web browser, open a new browser window,
and try logging in again.

• The following errors are reported in the SCBroker log file:

GenericLog     
GenericError    1       0       2005-11-14 10:50:14     (scbcomp.cpp
(416) err=7100015 sys=0) SBL-SCB-00015: The component is down or not
available on this server

GenericLog     
GenericError    1       0       2005-11-14 10:50:14     (scbcomp.cpp
(235) err=7100015 sys=0) SBL-SCB-00015: The component is down or not
available on this server

GenericLog     
GenericError    1       0       2005-11-14 11:06:18     (scbcomp.cpp
(822) err=7100011 sys=0) SBL-SCB-00011: Failed to connect to pipe
(SEBL_11_5316) on process 5316.

GenericLog     
GenericError    1       0       2005-11-14 11:06:18     (scbcomp.cpp
(416) err=7100011 sys=0) SBL-SCB-00011: Failed to connect to pipe
(SEBL_11_5316) on process 5316.

• Depending
  on the Siebel version you are using, you may encounter either a core
  dump or the process being frozen in the shutdown state. In the latter
  case you will need to kill it from the shell to get the process
  released.  Here is an example of an output from a process that is hung
  with the following call stack:

4017:   siebmtshmw /siebel/osprd/siebsrvr/admin/osPRDent.siebsrvr1.shm 11 6150

-----------------  lwp# 1 / thread# 1  --------------------

 7daa58f4 lwp_park (0, ffbfeae8, 0)

 7daa2b9c cond_wait_queue (ba25f0, 7dab8b48, 0, 0, 7d790000, 7dab8000) + d4

 7daa3114 cond_wait_common (0, ba25d8, ffbfeae8, 0, 0, 437afb47) + 1d8

 7daa35a4 _cond_timedwait (ba25f0, ba25d8, ffbfec18, 0, 0, 0) + 1f0

 7daa35d8 cond_timedwait (ba25f0, ba25d8, ffbfec18, 0, 0, 0) + 18

 7daa3618 pthread_cond_timedwait (ba25f0, ba25d8, ffbfec18, 10624c00, 0, 0) + c

 7fa1291c __1cMosaWaitEvent6Fpvi_i_ (ba25d0, 1388, 0, 0, ba25d8, 0) + 188

 00031600 __1cNsmiMainThreadIShutdown6M_v_ (695f38, a0, 1, 86070, 688, 400) + 50

 0002f7c8 __1cNsmiMainThreadIMainLoop6M_i_ (695f38, 1b7740, 0, 0, 7bd3080, 5e) + d00

 0002a754 wmain    (9, 12f100, 0, 0, 0, 0) + 274

 0005dd60 main     (ffbff174, 86070, ffbff19c, 7e157380, 74c, 400) + 94

• The object manager’s listener thread session log file contains the following error:

SisnTcpIp     
SisnSockError     1     0     2005-09-09 12:26:29     1:
[LOCALTRANS-server] accept() failed during get conn request (error=9602,
sys=4)

GenericLog    
GenericError      1     0     2005-09-09 12:26:29     (smimtsrv.cpp
(1538) err=2100006 sys=0) SBL-SMI-00006: Internal: SISListen failed

You
can identify the listener thread session log file by searching for the
pattern “ 1 “ in the header line of all log files by running the
following command:

head -1 *.log | grep “ 1 “

The output should return something like the following:

1021 2005-10-05 20:55:19 0000-00-00 00:00:00 +0200 00000000 001 003f 0001 09 FINSObjMgr_deu 16434 23184 1

Workaround or Resolution

If you are running on Siebel Maintenance Release 7.7.2.x, the above behavior has been fixed in Siebel Fix Pack version 7.7.2.5.

For other Siebel versions, you should perform the following analysis:

1. Check the SCBroker log file and confirm if it contains the SBL-SCB-00011 error.

2. Check the Object Manager listener log file and confirm if it contains the SBL-SMI-00006: Internal: SISListen failed error.

3. Check for the following:

1. If the process is hanging, run the command below and see if the call stack matches the one documented in this Alert.

pstack <PID_OF_OM>

          or

2. If
   the process has generated a core dump, run the command below and see if
   the thread# 1 is showing on the top of the stack trace as documented
   above.

pstack core

If
the errors plus call stack matches all three scenarios, then you should
apply the Siebel Fix Pack version 7.7.2.5 on top of Siebel Maintenance
Release 7.7.2.

If
you experience similar behaviors described in this Alert, but all the
above 3 scenarios do not match, then please log a new service request.
Provide all the relevant Siebel log files, pstack of hung processes or
crashed processes (core file).

For additional information about troubleshooting Server components behaviors on UNIX, refer to Document 477520.1 and Document 478027.1.

Maintenance Release Number

Please click on the link below to view the current status of the Change Request and associated Fix Requests:

Bug 10501186 has been fixed on version 7.7.2.5 and 7.8.2.2

Applies to:

Siebel CRM - Version: 8.1 [21039] and later   [Release: V8 and later ]

Information in this document applies to any platform.

Server busy error with custom OM -ACSWMTObjMgr_enu

Symptoms

When try get the Siebel login by using custom component - customer Object Manager -ACSWMTObjMgr_enu ,we get following error



http://10.203.168.143/acswmt_enu/start.swe?" URL getting the following error message:

The server you are trying to access is either busy or experiencing
difficulties. Please close the Web browser, open a new browser window,
and try logging in again.[23:24:13]



In OM logs, these errors were found



 2009-10-11 23:07:37 Login failed for Login name : anonemp

 2009-10-11 23:07:39 (ctxtmgr.cpp (4504)) SBL-SVC-00208: Please login first



In SWSE logs, these errors where found



2009-10-11 22:53:16 ( (0) err=4653071 sys=0) SBL-SCB-00015: The component is down or not available on this server



2009-10-11
22:56:21 (smconn.cpp (283) err=1180866 sys=2321) SBL-NET-01218: The
connection was refused by server hcmblysun004a. No component is
listening on port 2321.



 2009-10-11 22:56:21 (ssmsismgr.cpp (773) err=3670019 sys=0) SBL-SSM-00003: Error opening SISNAPI connection.

 2009-10-11 22:56:21 (ssmsismgr.cpp (1761) err=3670022 sys=0) SBL-SSM-00006: Error while sending message to server.

 2009-10-11
22:56:21 (smconn.cpp (283) err=1180866 sys=2321) SBL-NET-01218: The
connection was refused by server hcmblysun004a. No component is
listening on port 2321.

Changes

Created custom Object manager -ACSWMTObjMgr_enu

Cause

Incorrect value for Siebel Repository File parameter





.

Solution

Setting the correct value for Siebel Repository File parameter resolved this issue.

Applies to:

Siebel System Software - Version: 7.8.2.2 [19219] and later   [Release: V7 and later ]

z*OBSOLETE: Microsoft Windows Server 2003

Product Release: V7 (Enterprise)

Version: 7.8.2.2 [19219]

Database: Oracle 9i

Application Server OS: Microsoft Windows 2003 Server SP1

Database Server OS: Sun Solaris 9



This document was previously published as Siebel SR 38-3035052921.

Symptoms

SBL-SVR-01015, SBL-SVR-01043, SBL-SMI-00033, SBL-NET-01034, SBL-NET-01033, SBL-ADM-01044, SBL-SCB-00015, SBL-SCB-00011Hi,



Each night a cold DB backup is taken on each environment. This is affecting a number
of components on the Siebel server:



Transaction Processor

Transaction
Merger

Transaction Router

Server Tables Cleanup



These components need to be
restarted each morning as the backup is affecting them. I decided to use scripts to shutdown the
siebel server 15 mins before backup is taken and another to restart it in the morning. I have
tested these script on one environment and this works fine. On another environmnet the shutdown
script appears to hang (i.e batch job does not complete). All the parameters are correct as the
command would not run otherwise. I have attached the script and the log files at the time it
executed (01:45). As you can see from the log file at 06:00 created after the startup script is
executed, the server is still shutting down. Can you please inform me what the issue
is?



Kind regards

Cause

Configuration/ Setup

Solution

Message 1

For the benefit of other readers:



The customer's script:



D:\sea78\test\siebsrvr\BIN\srvrmgr /g UKGateway /e siebtest /s SiebTest1 /u sadmin /p password /c "shutdown appserver SiebTest1"



In review of the customer's component log files, we could see that there
was a problem with one or more components shutting down. As stated in
SupportWeb SR 38-1147657941 [Siebel Servers stick in "Shutting down"
state], the "shutdown appserver" command issued from the Server manager
command-line is a synchronous process, thus the command will block until
all tasks in the server have exited. There could be a problem stopping
one or more components, caused either by the components themselves, or
caused by the synchronous nature of the shutdown process (ie. one
component is waiting on a response from another component, or the Siebel
server - that may have already been shutdown).



The customer was recommended to use the Windows command, net stop. The net stop command is used as follows:



net stop <service>



For example:



net stop SiebSrvr



Which would stop the service called SiebSrvr.



Assuming you do not have any problem stopping your Siebel Server service
using the Windows' Services utility, then you should not experience a
problem with using the net stop command to stop the Siebel Server
service. You can schedule the command using the Windows command, at.



This information resolved this for the customer.



Regards,

SBL-SCB-00011: Failed to connect to pipe

Applies to:

Siebel System Software - Version: 7.7.2.2 [18356] and later   [Release: V7 and later ]
z*OBSOLETE: Microsoft Windows Server 2003

Product Release: V7 (Enterprise)

Version: 7.7.2.2 [18356]

Database: Oracle 9.2.0.6

Application Server OS: Microsoft Windows 2003 Server SP1

Database Server OS: Microsoft Windows 2003 Server SP1



This document was previously published as Siebel SR 38-3194443421.

Symptoms

Customer is experiencing a recurring issue with their Web Client
environment. More times a day the following error is logged in the
SCBroker log file.



SBL-SCB-00011: Failed to connect to pipe (SEBL_11_4248) on process 4248



At this moment no users can logon anymore and the ones which are logged
on receive the error "The Siebel Server is experiencing problems".



By re-starting the EAFObjMgr_enu the issue gets resolved for the time
being. But it's unclear why this happens. I attached a full .zip file of
the \log directory. The SCBroker component has evtloglvl %=4. This is
not true for other components including the EAFObjMgr_enu. At this stage
I'm hessitant to increase the EAFObjMgr_enu log levels.



Number of users affected ± 40.







See attached the log files + evt files (nothing to report here).



Regards,

Cause

Change request 12-1GVIZVC

Solution

For the benefit of other users:



Customer encountered SCBroker hanging behavior as per Alert 1252. After
dumping the hanging corresponding object manager process using adplus,
the following call stack could be made reponsible for the hang:



A thread executing a workflow is dead locked in CSSUInboxTypeCacheMgr call:



sslcosa!osaWaitEvent+0x11

sslcsrcn!SrmSisMsgQ::GetMsg+0x17

sslcsrcn!SrmConn::GetResponeses+0x5a

sslcsrcn!SrmConn::SubmitSynch+0xa1

sslcsrcn!SrmConn::SendClientInfo+0xb3

sslcsrcn!SrmConn::Connect+0x307



Change request 12-1GVIZVC   has been created to develop a fix that is preventing the hang.

Applies to:

Siebel CRM - Version 8.1.1 SIA [21111] and later
Information in this document applies to any platform.

Symptoms

When the customer tries to login via the Web Client after the restart
of Siebel Server and Web Server services, IE throws following error
sometimes:

The server you are trying to
access is either busy or experiencing difficulties. Please close the Web
browser, open a new browser window, and try logging in again.

It will displays a login page if the customer press F5(refresh) button after the above error.

Cause

It was observed that Object Manager was neither crashed nor hang.

The SCBroker_xxx.log file logged following error but this Process ID 25780 was not logged on the enterprise log file:

SBL-SCB-00011: Failed to connect to pipe (SEBL_20_25780) on process 25780.

This Process ID was logged on the enterprise log file which was one generation earlier.



It was found that IE tried to connect to the Object Manager process
which was one generation earlier. Because the Siebel Server and Web
Server services were restarted while connecting the Web Client user and
the customer used same IE process which has old session information.

Reproducible steps were found with following steps:

1. Launch IE with the URL like http://<web server>/callcenter_enu and login via the Web Client.



2. While connecting the Web Client user, restart the Siebel Server and Web Server services.



3. Press F5 button on the same IE process with above 1. Then IE throws following error:

The server you are trying to access is either busy or experiencing
difficulties. Please close the Web browser, open a new browser window,
and try logging in again.



4. Press F5 button again on the same IE process with above 1. Then login screen displays successfully.

Since this phenomenon is neither a crash nor hang of Object Manager, you
can ignore the SBL-SCB-00011 error on the SCBroker_xxx.log file.

Solution

Please launch a new IE process and login via the Web Client after the restart of the Siebel Server and Web Server services.

Applies to:

Siebel System Software - Version: 7.7.2 [18325] and later   [Release: V7 and later ]
Information in this document applies to any platform.

Area(s):System Administration

Release(s):V7 (Enterprise), V7 (Professional), V8 (Enterprise), V8 (Professional)

Database(s):All Supported Databases

App Server OS(s):All Supported Platforms

Latest release tested against:V8 (Enterprise)

Keywords:hang, SCBroker, hung, SBL-SWP-00121, SBL-SCB-00011



This document was previously published as Siebel Alert 1252.

Description

The
Siebel Connection Broker (SCBroker) is a server component that provides intraserver load
balancing. SCBroker distributes session login requests across multiple instances of Application
Object Managers (AOMs) running on a Siebel Server. SCBroker logic will always route the request
to the AOM process that has the least number of running tasks. For details see Siebel Bookshelf
version 8.0 > Siebel Deployment Planning Guide > Siebel Architecture Overview > About
the Siebel Connection Broker.

 

Under
certain scenarios, an AOM process can be in a hanging state and is not able to acknowledge
connection requests from the SCBroker anymore. If this hanging AOM process happens to have the
least number of running tasks, SCB will attempt to route new session requests to this AOM process
repeatedly. This results in blockage of all new login requests on that Siebel Server.

 

Bug 10644728 has been logged to address the product enhancement request to
have the SCBroker ignore this unresponsive AOM process.

Likelihood of Occurrence

This
Alert is applicable if you are running Siebel Application Server 7.7, 7.8 , 8.0 and 8.1.

Possible Symptoms

The
following criteria all needs to be met and these are the symptoms that you may encounter if you
are experiencing this type of deadlock behavior:

 

• Every login
  attempt to a certain AOM on the affected Siebel Server is prompted with the message:

 

SBL-SWP-00121:
The server you are trying to access is either busy or experiencing difficulties. Please close the
Web browser, open a new browser window, and try logging in again.

              It takes approximately 60 seconds until the server busy message appears. 

• However
  session logins to other components are working without any problem.

 

• The AOM
  component in the server admin screen shows up as running on the affected server.

 

• No crashes
  for the AOM are reported in the enterprise logs. No core dump or crash.txt file is written on
  the affected server.

 

• AOM tasks
  might be staying in the state “Handling Logon” or ”Relogin”.

 

• The SCBroker
  log is continuously logging the following error message:

 

SBL-SCB-00011:
Failed to connect to pipe (SEBL_0_pid) on process pid

 

where
pid is the process id of the corresponding object manager process. This process id can
be found in the enterprise log to associate it with a certain AOM.

 

• The process
  with this pid can still be found in the process list of the operating system and
  corresponds to one of the AOM processes on the affected server.

 

• 
  NOTE: Applicable to Windows only. You may observe a crash.txt file, however
  there is no error message SBL-OSD-01000 being logged in the enterprise log and the crashing
  process is not restarted despite the AutoRestart parameter is set to true.

Workaround or Resolution

The
workaround to get the SCBroker process operative again is to terminate the hanging AOM process
manually. This can be done using the kill command on UNIX or the Task Manager on Windows.

 

Once
the offending process has been terminated, SCBroker will continue normal distribution of new
session requests across the remaining AOM processes.

 

To
analyze the root cause that is responsible for the AOM hang, please log a new Service Request
with Technical Support and provide the information as described in the following
references:

 

• 
  Document 478050.1 discuss how to troubleshoot hangs on Microsoft Windows.

 

• 
  Document 478027.1 discuss how to troubleshoot hangs on UNIX.

 

Two
scenarios that can result in SCBroker disruption and the corresponding resolution are already
described in the following references:

 

1. 
   Document 473809.1 discuss Object Manager hang behaviors on UNIX Platforms due
   to calls to malloc memory management function in the crash handler code.

 

2. 
   Document 473854.1 discuss unexpected UNIX process signals can cause MT Server
   to shut down or hang.

 

NOTE: Applicable to Windows only and any changes made to the Windows registry
should be performed only after taking appropriate back-ups:

 

1. Check
   the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug

 

2. If
   the Auto key is set to 0, a message box will be displayed by Dr. Watson prior to postmortem
   debugging.

 

3. In
   case Dr. Watson is enabled, this can cause the SCBroker to still route connection requests to
   the remaining fragment of the crashed process during the 5 minute period where Dr. Watson is
   waiting for the Visual notification being acknowledged.

 

4. The
   Auto key must be set to 1 to avoid the pop up boxes in the future.

Applies to:

Siebel CRM - Version 7.8.2 [19213] to 8.1.1 SIA [21111] [Release V7 to V8]
z*OBSOLETE: Microsoft Windows Server 2003

Product Release: V7 (Professional) and later

Version: 7.8.2 [19213]

Database: Microsoft SQL Server 2005

Application Server OS: Microsoft Windows 2003 Server

Database Server OS: Microsoft Windows 2003 Server



This document was previously published as Siebel SR 38-3304776353.





*** Checked for Relevance on 11-Sep-2012 ***

Symptoms

SBL-NET-01023, SBL-SCB-00011, SBL-SSM-00004, SBL-SSM-00006


We have launched Siebel today. Users are seeing 4 problems:



1. Server busy or problem experienced browser error

2. White screen with progress bar

3. Siebel splash screen with progress bar (no login box)

4. Users get into the system and then get booted out.



When first launched, we had serious avaikability issues as above.

We then changed the following settings:



1. MAX TASKS params for all object managers from 20 to 100 (as recommended in PRR) except fo UK OM which was changd to 200.

2. MAX TASKS for connection broker from 1 to 2.



Since these changes users have been able to access, but within last 1 hour we have again seen the same issues.

Cause

Encryption for the communication between Siebel web server and Siebel servers is set to MSCRYPTO

Solution

Message 1

As per the client description their siebel web client became unavailable at certain times.

After investigations the symptoms were found to be as follows:

On the siebel application server there was a siebmtshmw process that
became to use 100% CPU. This process did not became unresponsive but
instead to became idle when no requests where processed it went to
100%CPU. On a 4 processor box when there were 4 siebmtshmw processes
that went in this state the machine begun to respond very slow this
resulting in dropping new sessions and overall log responses to
incomming requests.

On the web server machine (IIS) there was present the same situation
(the siebel extensions out of process begun to consume 100%CPU)

The thread that consumed CPU was identified from the performance monitor
logs and then core dump where taken from these processes. The thread
was a communication thread between the siebel web extensions (SWSE) and
object manager (OM). The call stack for the corresponding thread was:



sslcnapi!compress_write+0xdb

sslcnapi!zlib_inflate+0x8d

sslcnapi!CSSSISConn::_DecodeRawMsg+0x158

sslcnapi!CSSSISConn::SisAsyncThreadMain+0x8d

sslcosd!OSDThreadStart+0x1c0 sslcosd!OSDNTThreadStart+0xc

msvcr70!beginthreadex+0xba

kernel32!GetModuleFileNameA+0xeb

...



The relevant line in the communication call stack is
CSSSISConn::_DecodeRawMsg. The customer had the encryption for the
communication set to MSCRYPTO. This encryption is not largely used or
recommended by Siebel. After setting the encryption to NONE the problem
did not re-occurred.



Resolution:



The customer choose to check if they need encrypted communication
between the web server and the application servers and if this is the
case use SSL instead.

Applies to:

Siebel Assignment Manager - Version: 7.7.2.6 SIA [18372] and later   [Release: V7 and later ]
Information in this document applies to any platform.

Symptoms

After encountering a cluster server crash which forced a restart of
the application server it was no more possible to invoke Assignment
Manager to Assign Service Requests.



Error reported includes:


[1] Could not route message to AsgnSrvr with registered key (null)
[2] no other server
[3] Stack trace:
Service [Synchronous Assignment Manager Requests].InvokeMethod(), Built-in function
BusComp [Service Request].BusComp_WriteRecord(), Line: 1568


From the logs:


ObjectMgr
SBL-SRB-00047: Could not route message to AsgnSrvr with registered key (null)

SRBroker
GenericLog
GenericError 1 0 2008-07-15 07:53:48 (scbcomp.cpp (822) err=7100011
sys=0) SBL-SCB-00011: Failed to connect to pipe (SEBL_32_2460) on
process 2460.

Cause

After a review of siebns.dat, the following section showed that the AsgnMgmt components  were 'Online' but not 'Enabled'
----------
[/enterprises/VFECRMPRD/servers/SIEBRPT01/component groups/AsgnMgmt]
Type=empty

[/enterprises/VFECRMPRD/servers/SIEBRPT01/component groups/AsgnMgmt/definition]
Type=empty

[/enterprises/VFECRMPRD/servers/SIEBRPT01/component groups/AsgnMgmt/definition/enable state]
Persistence=full
Type=string
Value="disabled"
Length=16

Details
in the Gateway server caches provides statuses and availability of each
server and it's components within the enterprise and this is what is
used at startup.

The Cluster Server crash corrupted the
siebns.dat and so after the restart, the corrupted values for the
assignment components 'State' were being used.

Solution

The corrupted siebns.dat file was amended by resetting the assignment manager components 'enabled state' to 'enabled'.


Example:


From:


[/enterprises/VFECRMPRD/servers/SIEBRPT01/component groups/AsgnMgmt/definition/enable state]
Persistence=full
Type=string
Value="disabled"
Length=16


To:


[/enterprises/VFECRMPRD/servers/SIEBRPT01/component groups/AsgnMgmt/definition/enable state]
Persistence=full
Type=string
Value="enabled"
Length=16


 Once all the changes have been made, the system was restarted in
the correct order to bring all components back  to an available state.


i.e.


a) Start Database


b) Start Gateway


c) Start Siebel Server

SBL-SCB-00008: Component (sccobjmgrsmc_enu*) is unavailable on this server

Applies to:

Siebel CRM - Version: 7.8.2.8 SIA [19237] and later   [Release: V7 and later ]
Information in this document applies to any platform.

Goal

The following errors are happening in the scbroker files -
err=7100008 sys=0) SBL-SCB-00008: El componente
(ecommwirelessobjmgr_esn) no estM-CM-!
disponible en este servidor - why ?

Solution

An1:
Hi Clemente,

Thank you for taking my call earlier.

As
discussed I believe your issue is related to the your VirtualServer
section in your lbconfig.txt containing servers where OM's are not
running.

The following 7.8 bookshelf reference explains how to
configure your lbconfig.txt and eapps.cfg to cater for this scenario
> Siebel System Administration Guide > Configuring the System
Architecture > Configuring Siebel Server Load Balancing >
Optimizing the Siebel Load-Balancing Performance.

Please review
this section and modify your lbconfig.txt and eapps.cfg accordingly. I
suggest you test this configuration on your pre-production environment
initially.

If you have any further questions on this please let me know.



Yes,
you only need servers using Application Object managers in these lines
which in your case I believe is EAI and ecommwirelessobjmgr_esn.

As
mentioned on the phone I strongly recommend that any changes to your
eapps.cfg/lbconfig.txt files be tested in pre-production prior to
rolling ou in production.

Applies to:

Siebel CRM - Version: 7.8.2.10 [19241] and later   [Release: V7 and later ]
Information in this document applies to any platform.

Goal

When attempting to load balance two new Siebel servers and deploy them in production,
the following errors occur in SCBroker log:

SBL-SCB-00008: Component (sccobjmgrsmc_enu*) is unavailable on this server.
SBL-SCB-00008: Component (sccobjmgrsmc_eng*) is unavailable on this server.

This
is a production environment where they are trying to deploy 2 new load
balanced servers. The existing servers are already load balanced and
that works ok.
The difference is that the keepalive url is set to
"/crmprod/SCCObjMgrSMC_eng" on the new LB server, in comparison with the
CRMHDLB (the one that is working) where the keepalive uri is set to
"/CRMPROD/scbroker".

Solution

After changing keepalive uri on the load balancer to point to SCBroker
no error is generated in the log files. In this case the root cause was
incorrectly configured uri for keepalive connection from the load
balancer.

SBL-SCB-00007: Enterprise name ( %1) in the request is invalid.

Applies to:

Error Message Area:Server Infrastructure - SCB

Version:Siebel 8.0 and later

Purpose

This document is intended to provide cause and corrective action
information about Siebel Error Message SBL-SCB-00007: Enterprise name (
%1) in the request is invalid.

Scope

This document is informational and intended for any user.

SBL-SCB-00007: Enterprise name ( %1) in the request is invalid.

Explanation

Enterprise name in connect string could be wrong.

Corrective Action

Check eapps.cfg configuration - make sure that enterprise name matches that of the server.

Applies to:

Product Release: V7 (Enterprise)

Version: 7.8.2.3 [19221]

Database: IBM DB2 8.2

Application Server OS: Microsoft Windows 2003 Server

Database Server OS: IBM AIX 5L 5.2



This document was previously published as Siebel SR 38-3365847407.

Symptoms

SBL-SCB-00007What is the error code SBL-SCB-00007. I was not able to retrieve anything from the search on
this from the knowledge base. In our production environment we get a error GenericLog
GenericError 1 0 2007-05-31 13:22:16 (scbcomp.cpp (235) err=7100007 sys=0) SBL-SCB-00007:
Enterprise name ( sbldev_ent) in the request is invalid in the scbroker.log file, it is becoming
a huge file. sblprod_ent is the enterprise name but not sure why and what is looking for
sbldev_ent which is the name for the development enterprise. I have attached the eapps.cfg file
for your reference. Please note we modifid the sections we use for production in the eapps.cfg
file from Dev environment and placed it in production. We use F5 bigip to load balance the siebel
app servers.

Thanks

Solution

Message 1

For the benefit of other readers:





The error code SBL-SCB-00007 means that the Enterprise name in connect
string could be wrong. Then you must check eapps.cfg configuration and
make sure that enterprise name matches that of the server.



However the eapps.cfg configuration file was configured properly and
also any request to sbldev_ent was not found in the Siebel Web Server
log files.



Siebel Technical Support suggested customer to install a network trace
utility or sniffer tool such as ethereal on the Siebel server and trace
the traffic to the SCBroker Port 2321 and after that customer was able
identify that even after bringing down the entire qa environment, the
port 2321 was getting traffic.



Customer worked with their network team to understand the source of the
traffic and they were able to identify that the BigIP health monitor was
not setup correctly to look at SBLQA_ENT and hence the errors on
SCbroker for invalid enterprise name.



Regards

Applies to:

Siebel CRM - Version: 8.1.1 SIA [21111] and later   [Release: V8 and later ]
Information in this document applies to any platform.

Symptoms

Environment:

-------------------

Siebel 8.1.1 SIA [21111], HP-UX 11.11 Itanium (32-bit), Test environment



Statement of Issue:

-----------------------------

Unable to access the Siebel application using Thin client, giving server busy error message



Steps:

---------

1) Launch Internet explorer

2) Enter thin client URL & hit enter



Error:

------

SBL-SCB-00007: Enterprise name ( (null)) in the request is invalid.



Business Impact:

-------------------------

Unable to make use of Siebel application

Changes

Reconfigured the SWSE with incorrect SC Broker port number

Cause

Incorrect SC Broker port number mentioned while reconfiguring SWSE

Solution

1) Stop Web server

2) Take a back up of existing eapps.cfg & eapps_sia.cfg files

3) Replace the in correct SC Broker port number to correct one [find and replace all incorrect values]

4) Save files

5) Restart the web server

References

NOTE:493180.1 - SBL-SCB-00007