יום שישי, 5 באפריל 2013

SBL-SVR-00052: Internal: Invalid proc handle



Applies to:


Siebel CRM Service, SPE - Version 8.0 [20405] to 8.0 [20405] [Release V8]
Information in this document applies to any platform.



Symptoms








Statement of what the issue is



After a fresh installation, the Server Request Broker component fails to initialize at startup



ERROR generated in Siebel Enterprise log

-------------------------------------------------

ServerLog ComponentUpdate 2 0000290248210580:0 2008-05-07 14:18:31 SCBroker INITIALIZED Component has initialized.



GenericLog GenericError 1 00007988482104d8:0 2008-05-07 14:18:34
(scirkey.cpp (1142) err=1310772 sys=2) SBL-SVR-00052: Internal: Invalid
proc handle



ServerLog ProcessExit 1 00007988482104d8:0 2008-05-07 14:18:34 SRBroker 3584 TERMINATED Process 3584 was terminated



GenericLog GenericError 1 0000290248210580:0 2008-05-07 14:23:26





ERROR generated in Server Request Broker log

---------------------------------------------------------

ServerLog LstnObjInherit 3 0000000248210e00:0 2008-05-07 14:18:26 Inherited listening object for port 49177



ServerLog LstnObjPrivCreate 3 0000797f482104d8:0 2008-05-07 14:18:26 Created port 49180 for Server Request Broker



DBCLog DBCLogError 1 0000797f482104d8:0 2008-05-07 14:18:29 [DataDirect][ODBC Oracle driver][Oracle]ORA-12541: TNS:no listener





GenericLog GenericError 1 0000797f482104d8:0 2008-05-07 14:18:29
(srbthrd.cpp (3997) err=2097168 sys=0) SBL-SRM-00016: Unable to
initialize the Database environment -- Unable to connect to DB (data
ops)



GenericLog GenericError 1 0000797f482104d8:0 2008-05-07 14:18:29
(srbmtsrv.cpp (86) err=2097168 sys=0) SBL-SRM-00016: Unable to
initialize the Database environment -- Unable to connect to DB (data
ops)



SrbLayerLog Error 1 0000797f482104d8:0 2008-05-07 14:18:29 Main Init fails



GenericLog GenericError 1 0000797f482104d8:0 2008-05-07 14:18:29
(smimtsrv.cpp (1187) err=2097168 sys=0) SBL-SRM-00016: Unable to
initialize the Database environment -- Unable to connect to DB (data
ops)



SmiLayerLog Error 1 0000797f482104d8:0 2008-05-07 14:18:29 Terminate process due to unrecoverable error: 2097168. (Main Thread)



Cause


After reviewing the logs provided, it was determined that possible
problem areas include (a) an incorrectly defined ODBC data source and/or
(b) problems with the the Listener configuration; although wider
Database connectivity problems cannot be discounted.



DBCLog DBCLogError 1 0000797f482104d8:0 2008-05-07 14:18:29 [DataDirect][ODBC Oracle driver][Oracle]ORA-12541: TNS:no listener


"ORA-12541: TNS:no listener" is usually logged when a connection
request can not be completed because the listener is not running (or is
simply unavailable) at the specified location - see Note:21475.1 on
Metalink



Corrective Action is described as follows:

Ensure that the supplied destination address matches one of the
addresses used by the listener - compare the TNSNAMES.ORA entry with the
appropriate LISTENER.ORA file (or TNSNAV.ORA if the connection is to go
by way of an Interchange). Start the listener on the remote machine.



Solution




Based on a review of the logs recently uploaded, the most probable cause
of the initialization problem is an incorrectly defined ODBC Data
Source or a misconfigured listerner



It would therefore be of some help it you could carry out the following
measures and provide proof on the results obtained at each stage (screen
shots, *.cfg file (3)), if further assistance is required.



(1) Verify the ODBC Data Source is correctly defined as shown in this
bookshelf reference: 'Siebel Installation Guide for Microsoft Windows
> Configuring Siebel Enterprise Server and Related Components >
Verifying the ODBC Data Source > Verifying the ODBC Data Source for
Oracle'



and/or



(2) Test connectivity to the database, using the ODBC DataSource tool
and also verify the hostname of the Server hosting the Database Listener
by comparing TNSNAMES.ORA entry with appropriate LISTENER.ORA file.



and/or



(3) Ensure that the ODBC Data Source name also matches the Server connect string specified in siebel.cfg on the Siebel Server.










Applies to:


Siebel System Software - Version 8.0.0.8[20430] to 8.1.1.7 [21238] [Release V8]
Information in this document applies to any platform.



Symptoms



Customer had installed 8.0.0.9 patchset to production
enviroment. Previously customer had Siebel 8.0.0.8 version and was
installed on the IBM AIX 5.3 (64-bit). After installation of 8.0.0.9
patchset siebel server was not coming up.


Error:
-------
Siebel server Logs:
--------------------
GenericLog
GenericError 1 000069d14e0b30c0:0 2011-06-29 18:15:09 (sccnmsys.cpp
(1305) err=2883609 sys=0) SBL-SCC-00025: No value found in the Gateway.
Default value in the repository is used.
GenericLog GenericError 1
000069e04e0b30c0:0 2011-06-29 18:15:09 (sccnmsys.cpp (1305) err=2883609
sys=0) SBL-SCC-00025: No value found in the Gateway. Default value in
the repository is used.


Enterprise Logs:
------------------
GenericLog
GenericError 1 000076ee4deb1046:0 2011-06-05 13:31:39 (scirkey.cpp
(1142) err=1310772 sys=0) SBL-SVR-00052: Internal: Invalid proc handle
ServerLog
ProcessExit 1 000076ee4deb1046:0 2011-06-05 13:31:39 SRBroker 594106
SBL-GEN-00255 Process 594106 exited with error - Internal: Error during
exec()




Changes


Applies patchset 8.0.0.9 on top of 8.0.0.8.



Cause


After Verification it was found installable library files from
previous installation 8.0.0.8 got corrupted and hence installation of
8.0.0.9 is not getting through.

Further verification showed,
release timestamp for some library files from 'siebsrvr/lib' were not
set to patch release date but patch install date.
In Ideal case the
timestamp should correspond to patch release date, thing is that the
release time stamp is only set at the very end. If something breaks
before then files remain on disk with install date.

In case of
customers environment, some library files has timestamp of April2011
which is patch installation date, in correct scenario it should have
been patch 8.0.0.8 release date (Oct2009).
Which shows incorrect deployment of 8.0.0.8 patchset.



Solution


Below are some prominent steps for re-installation of gateway and Siebel Servers:

Reinstalling the Gateway Server:
-----------------------------------------------------------
1) Stop the Gateway service
2) Uninstall the Gateway server.
3) Delete the Gateway server directory structure.
4) Reboot the machine.
5) Install the Gateway server.

Reinstalling the Siebel Server:
-----------------------------------------------------------
1) Make backups of all the .cfg files present in \siebsrvr\bin and the siebns.dat file present in \gtwysrvr\ADMIN.
2) Make backup of the .srf file present in \siebsrvr\OBJECTS.
3) Make backups of the following folders present in the \siebsrvr directory to take care of any customizations:
IDCENTRIC, MSGTEMPL, REPORTS, WEBTEMPL.

4) I am not sure if you have remote users in your environment, If there
are remote users working off this application server, backup the
following folders to avoid reextracting them:
DOCKING, DBTEMPL
5) Stop the Siebel server service
6) Uninstall the Siebel server
7) Delete the Siebel server directory structure.
8) Remove Siebel Server from the Gateway by removing configuration from wizard.
9) Reboot the machine.
10) Start the Gateway server service if not running already.
11) Install the Siebel server using same installation parameters as before.
12) Copy the backed up folders, .cfg files and the .srf file to the corresponding directory where it belongs.
13) Stop and restart the Gateway and Siebel server services.
14) From the Server Administration screen enable the needed component groups.

There is a note out there on MOS with the correct steps. It refers to Windows but same steps apply to AIX as well:
Reinstalling the Siebel Servers (Doc ID 476317.1)

Note:
This solution was specific to one environment/scenario, there is
possibility that same solution may not work for other environments.











Applies to:


Siebel System Software - Version: 8.0 [20405] and later   [Release: V8 and later ]
z*OBSOLETE: Microsoft Windows 2000

Product Release: V7 (Enterprise)

Version: 7.5.3.15 [16279]

Database: Oracle 9.2.0.8

Application Server OS: Microsoft Windows 2000 Server SP 4

Database Server OS: Red Hat Linux 4.0



This document was previously published as Siebel SR 38-3355796733.



Symptoms


Customer reported the following:


We just completed the Siebel upgrade from version 7.5.3.15 to version
8.0. Presently we are testing the application and when we go to
Admin-> Server Management screen, we are sometimes getting the
following errors:
>>>
We dedected an Error which may have occurred for one or more of the following reasons:
28*SBL-SCM-00028: Key not found1*012*sccconfg.cpp4*26880*0*0*0*
<<<

and
>>>
We detected an Error which may have occured for one or more of the following reasons:
SBL-SCM-00028: Key not found
<<<

>>
>>



Cause


The Problem was caused by a corrupted enterprise entry in the enterprise configuration file siebns.dat.
As
a result of this case, document bug 10524274 was logged to address the
fact that there is no explicit documentation in the Siebel CRM bookshelf
how to remove an enterprise from the configuration.

SBL-SVR-00052


Solution


For the benefit of other readers:

After reviewing the siebns.dat
file it could be noticed that there are two enterprise entries. One
enterprise called 'PRODUCTION' and the other one 'test'. The 'test'
enterprise had no sub key entries which is an indication that this
enterprise entry is corrupt.

To remove the corrupted enterprise
entry from the gateway name server configuration and thus from the
siebns.dat file the customer used the following steps:
1> stopped all SWSE web server services
2> stopped all Siebel server services
3> used the enterprise configuration wizard to remove the enterprise 'test'.
4> restarted the gateway name server service
5> started the Siebel server services
6> started the SWSE web server service
After
that the customer logged in to the Siebel Callcenter application using a
Siebel web client (thin client) and was able to navigate to the Siebel
system administration screens.










Applies to:


Siebel CRM - Version: 8.1.1 SIA [21111] and later   [Release: V8 and later ]
Information in this document applies to any platform.

***Checked for relevance on 17-Feb-2012***



Symptoms


The Gateway daemon for Siebel Industry Applications version 8.1.1
running on IBM AIX 5.3 against Oracle 11g shows high CPU. This can be
observed after starting a Siebel server, which has a duration of 2-3
minutes. After the Siebel server has started the CPU of the Gateway
daemon reduces and stabilizes, however the Siebel Workflow components
fail after startup. If these failed components are restarted manually
via Siebel Server Manager then the CPU increases on the Gateway as
before.

ERROR MESSAGES


  1. SBL-DAT-00173: An invalid key was entered.

  2. SBL-NET-01218: The connection was refused by server %1. No component is listening on port %2.

  3. SBL-SRB-00040: Cannot open asynchronous SISNAPI connection.

  4. SBL-GEN-05009: Unable to connect to the gateway server.

  5. SBL-SVR-00052: Internal: Invalid proc handle





Cause


Issue caused by extremely large SIEBNS.DAT which can be seen by the high
CPU in the gateway process originating from the reads performed by the
NSClient.

Bug 10643323 has been logged to address a Product
Enhancement Request so that the size of the SIEBNS.DAT does not grow
excessively and if it does then to provide means to reduce the size - in
particular removing event log level entries.

The large amounts of reads being performed by the NSClient are directly related to the high CPU observed on the gateway


Solution


To reduce the size of the SIEBNS.DAT, the following can be employed to resolve the issue:
1. Restore backup of SIEBNS.DAT after changing event log levels for large amount of server components
2. Remove unnessary component definitions










Applies to:


Siebel Assignment Manager - Version 8.0.0.5 SIA [20420] and later
Information in this document applies to any platform.



Symptoms


Customer was calling Assignment Mgr component from Siebel workflow
using synchronous 'Server Request'. They were seeing Assignment Mgr
failures and errors in Server Request Broker logs.



The issue / error message still happened even after the enterprise
restart/reboot this morning. This issue had caused Leads exceptions when
ever they ran the Assignment Manager job with a couple of hundred
records to over a thousand. The issue had been intermittent and not
reproducible in any other environments.



1) The following error was found in the Enterprise log error:



“SBL-SVR-00052: Internal: Invalid proc handle”. Then AsgnSrvr is
terminated. So it never stays online. Thus SR Broker can never see it,
and Assignment Manager does not work.



2) The following errors were prevalent in the Workflow, SRBroker and Assignment Manager logs:



"LM Automation Process Failed: Error invoking service 'Server Requests',
method 'SubmitRequest' at step 'Invoke Assignment
Rules.'.(SBL-BPR-00162)

--

"GenericError 1 0000123d4b9425bc:0 2010-03-12 00:10:30 (asgnsrvr.cpp
(914) err=851969 sys=0) SBL-OSD-00001: Internal: Function call timed out
(0)"



SBL-SRB-00061: Process AsgnSrvr on Siebel Server AMSPROINT1 terminated

SBL-BPR-00162: Error invoking service 'Server Requests', method 'SubmitRequest' at step 'Invoke Assignment Rules.'.



“13400: [TCPIP-server] bind() to INADDR_ANY:49152 failed for sd=5464
(err=10048 | Address is already in use (TCP/IP address/port).)”



“SBL-SVR-09016: Failed to get task instance: task number 1516240902. The task may have exited or does not exist.”







Cause


There is one AsgnSrvr parameter caught Oracle’s attention: "Refresh
people skills interval" = 60 (seconds). The setting is too low for
typical usage. If the refresh process takes too much time to complete,
it may prevent AsgnSrvr to proceed. it may worth for further
investigation.



This is parameter is also called "MaxSkillsAge". The customer had the
AsgnSrvr component parameter 'MaxSkillsAge=60': 60 seconds is too short.
While reloading Skills, AsgnSrvr cannot process any request. If the
skill data doesn't change frequently, 1 day may be more appropriate
(which would be 86,400 seconds), depending on how often employee skills
are updated.







Solution


After changing MaxSkillsAge parameter to 0, customer did not see
Assignment Manager error in any of the servers. They set it to zero
instead of 1 day because the customer uses the Release button in the
application that clears the rules from the system cache, re-evaluates
the skills and regenerate the rules.



Questions to PM:



Question 1: Could you please double confirm (if required) from
Engineering that Clear Rules will re-evaluate the skills too? This will
confirm that the MaxSkillsAge parameter can be set to 0 (default value -
disabled).

Answer: Yes. By clicking the Clear Rules (OOB it is called “Release”
button), the rulecache will be regenerated and the skills and skill
items will be reloaded into memory from the database.



Question 2: If the MaxSkillsAge is disabled (value set to 0), how does
it impact the Dynamic Assignment functionality where the user does not
get to control "Clear Rules" button in the application? If the Skills
are modified on daily basis or hourly basis and MaxSkillsAge is
disabled, how are the skills evaluated in Dynamic Assignment jobs?

Answer: When AsgnSrvr component is started the skills will be loaded
into memory, and this will be used for Dynamic assignment until there is
a reload either due to “Clear Rules” button or because MaxSkillsAge is
set to a non zero value. So if you set MaxSkillsAge to zero and the
Clear Rules button is not clicked then the values stored in memory
during startup will be used.










Applies to:


Siebel Finance - Version: 8.1.1.3[21219] and later   [Release: V8 and later ]
Information in this document applies to any platform.



Symptoms


When "start_server all" command is excuted on already runnign server, Its stopping the already running server on AIX.

Siebel Server "edcasbld309" (Enterprise "ccp8s1") is already running.
started at Thu Oct 20 15:02:34 2011, pid: 1155072, autostart: no




Cause


Tested it internally by running start_server all command on already running server on AIX,
And when i checked the enterprise log, components are terminating with below error in enterprise log.

GenericLog
GenericError 1 00000f6f4ec64046:0 2011-11-21 21:49:38 (scirkey.cpp
(1142) err=1310772 sys=0) SBL-SVR-00052: Internal: Invalid proc handle
ServerLog ProcessExit 1 00000f6f4ec64046:0 2011-11-21 21:49:38 SRBroker 835728 TERMINATED Process 835728 was terminated












Applies to:


Siebel CRM - Version 8.1.1.4 SIA [21225] and later
Information in this document applies to any platform.



Symptoms


The SRBroker component is crashing at Siebel Server startup


The Siebel Enterprise log shows the crash when the Siebel server is starting :


ServerLog ComponentUpdate 2 00002e305004003e:0 2012-07-17 00:02:40
SrvrSched INITIALIZED Component has initialized (no spawned procs).

ServerLog ProcessCreate 1 00002e305004003e:0 2012-07-17 00:02:40
Multithreaded-Server-Prozess (OS-PID= 13959346 ) für SRBroker erstellt

ServerLog ProcessCreate 1 00002e305004003e:0 2012-07-17 00:02:40 Server-Prozess (OS-PID= 24051950 ) für SCBroker erstellt

ServerLog ProcessCreate 1 00002e305004003e:0 2012-07-17 00:02:40 Server-Prozess (OS-PID= 28770536 ) für SCBroker erstellt

ServerLog ComponentUpdate 2 00002e305004003e:0 2012-07-17 00:02:45 SCBroker INITIALIZED Component has initialized.

GenericLog GenericError 1 00002e315004003e:0 2012-07-17 00:02:49
(scirkey.cpp (1142) err=1310772 sys=0) SBL-SVR-00052: Intern: Ungültiger
Prozess-Handle

ServerLog ProcessExit 1 00002e315004003e:0 2012-07-17 00:02:49 SRBroker
13959346 SBL-OSD-02011   Process 13959346 exited with error - Prozess
wegen einer Segment-Verletzung beendet (SIGSEGV)

ServerLog ProcessCreate 1 00002e305004003e:0 2012-07-17 00:02:50
Multithreaded-Server-Prozess (OS-PID= 39518450 ) für SRBroker erstellt

ServerLog ComponentUpdate 2 00002e305004003e:0 2012-07-17 00:03:00 SRBroker INITIALIZED Component has initialized.




Looking at the core file for this crash in dbx debugger ... crashing thread:


(dbx) where

pth_signal.pthread_kill(??, ??) at 0xd05098c0

pth_signal._p_raise(??) at 0xd0508d28

raise.raise(??) at 0xd01373e0

_SigBusSegvIotHandler(int,int,__sigcontext*)(??, ??, ??) at 0xd0f18020

CSSNSClient::SubTreeRead(const wchar_t*,NSScope,NSReqObj*&,unsigned int&)(??, ??, ??, ??, ??) at 0xd2492588

scmNSClient::ExecRequest(scmRequestObj*,scmExecReqFlags)(0xd05d9bbc, 0x3002cd50, 0x3002ce78) at 0xd1da51b8

sccQueryObj::Execute()(??) at 0xd282905c

sccQueryObj::FindObj(const wchar_t*,sccObject*&)(??, ??, ??) at 0xd2828be8

sccComponent::FindConnStr(const wchar_t*,sccConnectStr*&)(??, ??, ??) at 0xd27f9c04

SRBRouter::_GetOneServerInfo(sccServer*,NSSrvrInfo*)(??, ??, ??) at 0xd3e1e358

SRBRouter::_GetAllServerInfo(sccEnterprise*,SRBQueue*&)(??, ??, ??) at 0xd3e1c3dc

SRBRouter::CacheNS(int)(??, ??) at 0xd3e07c20

SRBRouter::Init()(??) at 0xd3e071f4

SRBMTServer::MainInit()(??) at 0xd388d390

smimtsrv.smiMainThread::CompMainInit()(??) at 0x100135f0

smimtsrv.smiMainThread::CompMainInit()(??) at 0x1000a028

smiMainThread::Startup()(??) at 0x10006cb8

wmain(??, ??) at 0x100035f4

main(??, ??) at 0x100032c4




Threads in the debugger ...


(dbx) th

thread  state-k     wchan    state-u    k-tid   mode held scope function

>$t1     run                  running  93717039     k   no   sys  pthread_kill

$t2     run                  running  79298627     u   no   sys  select

$t3     run                  blocked  41353839     u   no   sys  _event_sleep

$t4     run                  blocked  78315649     u   no   sys  _event_sleep

$t5     run                  blocked  87818473     u   no   sys  _event_sleep

$t6     run                  blocked  12583097     u   no   sys  _event_sleep









Cause


This crash occurs when multiple Siebel Servers are started in short
order. A race condition is leading to a crash of the SRBroker on some of
the Siebel Servers.






Solution


A bug has been raised for the forthcoming 8.1.1.9 Fix Pack:


BUG 13976875: SRBROKER CRASHES AT STARTUP




A bug has been raised for a Quick Fix for the 8.1.1.8 Fix Pack :


BUG 14380887: SRBROKER CRASHES AT STARTUP




The current workaround is to stop and restart the Siebel Server if this
SRBroker crash is seen when the server starts.  The restart of the
server at this point means that the server is not in the short order
sequence, hence the SRBroker should not encounter this defect.



















אין תגובות:

הוסף רשומת תגובה