Changes for page Troubleshooting Frozen Deployed Instances
Last modified by David Avendasora on 2010/11/30 06:43
From version 3.1
edited by Quinton Dolan
on 2007/07/14 21:28
on 2007/07/14 21:28
Change comment:
There is no comment for this version
To version 2.1
edited by smmccraw
on 2007/07/08 10:33
on 2007/07/08 10:33
Change comment:
There is no comment for this version
Summary
-
Page properties (2 modified, 0 added, 0 removed)
Details
- Page properties
-
- Author
-
... ... @@ -1,1 +1,1 @@ 1 -XWiki. qdolan1 +XWiki.smmccraw - Content
-
... ... @@ -1,10 +1,5 @@ 1 1 This article was written by Andrew Lindesay (http:~/~/www.lindesay.co.nz) around February 2005. It first appeared as LaTeX PDF and has been transcribed into this Wiki. You use the information contained in this document at your own risk. Please contact the author if you feel there may have been an error in the conversion to Wiki markup. 2 2 3 -|=Contents 4 -| 5 - 6 -{{toc style="disc"}}{{/toc}} 7 - 8 8 = Applicability = 9 9 10 10 The material discussed here has been used with WebObjects 5 and Java 1.4 on MacOS-X Server. It may or may not work on older or newer versions of WebObjects or Java. It is strongly suggested that you test this on a non-production server first. Note that there may be security issues with this technique if your system is exposed on the internet. ... ... @@ -21,24 +21,25 @@ 21 21 22 22 = Before you Start = 23 23 24 -A WebObjects system which is deployed in the traditional manner consists of a number of copies of the program running separately, each carrying some of the inbound load from users. Each of these 'copies' is termed an instance. The ##SiteConfig.xml## file defines the instances. This configuration file is located at the following place in your MacOS-X Server's file system. 19 +A WebObjects system which is deployed in the traditional manner consists of a number of copies of the program running separately, each carrying some of the inbound load from users. Each of these 'copies' is termed an instance. The ##computer code|SiteConfig.xml## file defines the instances. This configuration file is located at the following place in your MacOS-X Server's file system. 25 25 26 -{{ noformat}}21 +{{code}} 27 27 28 28 /Library/WebObjects/Configuration/SiteConfig.xml 29 29 30 -{{/ noformat}}25 +{{/code}} 31 31 32 -Before you modify it, make a backup of the ##SiteConfig.xml## file in case anything goes wrong. 27 +Before you modify it, make a backup of the ##computer code|SiteConfig.xml## file in case anything goes wrong. 33 33 34 34 = Setup = 35 35 36 -The instances are modified such that they are able to be connected to remotely using the ##jdb## debugging tool. Some "additional arguments" need to be inserted into the configuration in the ##SiteConfig.xml## file for each instance in order to achieve this. These additional arguments are inserted as shown below in the text of the element ##additionalArgs##. You will need to choose a different address for each instance ~-~- choose addresses from 8000 - 8999. This is a TCP/IP port. Note that the additional arguments should all appear on one continuous line. The author has split this up here to improve readability. 31 +The instances are modified such that they are able to be connected to remotely using the ##computer code|jdb## debugging tool. Some "additional arguments" need to be inserted into the configuration in the ##computer code|SiteConfig.xml## file for each instance in order to achieve this. These additional arguments are inserted as shown below in the text of the element ##computer code|additionalArgs##. You will need to choose a different address for each instance ~-~- choose addresses from 8000 - 8999. This is a TCP/IP port. Note that the additional arguments should all appear on one continuous line. The author has split this up here to improve readability. 37 37 38 -{{code value="xml"}}33 +{{code}} 39 39 40 40 ... 41 41 <instanceArray type="NSArray"> 37 +{panel} 42 42 <element type="NSDictionary"> 43 43 <id type="NSNumber">1</id> 44 44 <port type="NSNumber">2001</port> ... ... @@ -47,14 +47,17 @@ 47 47 -Xdebug 48 48 -Xrunjdwp:transport=dt_socket,address=8121,server=y,suspend=n 49 49 </additionalArgs> 46 +{panel} 50 50 ... 48 +{panel} 51 51 </element> 50 +{panel} 52 52 53 53 {{/code}} 54 54 55 -The ##element## tag will repeat here for all the instances inside the ##instanceArray## tag. 54 +The ##computer code|element## tag will repeat here for all the instances inside the ##computer code|instanceArray## tag. 56 56 57 -The ##id## tag gives the instance number and you need to remember the mapping from the instance number to the ##address## in the additional arguments. Jot this information down on a piece of paper. For example, one can see above that the instance 1 is mapped to address 8121. 56 +The ##computer code|id## tag gives the instance number and you need to remember the mapping from the instance number to the ##computer code|address## in the additional arguments. Jot this information down on a piece of paper. For example, one can see above that the instance 1 is mapped to address 8121. 58 58 59 59 Now restart your instances. 60 60 ... ... @@ -66,39 +66,39 @@ 66 66 67 67 Once you have the instance number, using your instance to address mapping from the setup, identify the address you want to connect to. 68 68 69 -Use the ##jdb## command line tool that comes with the java environment to connect to the instance and debug it. To do this, enter a command of the following form on the application server. 68 +Use the ##computer code|jdb## command line tool that comes with the java environment to connect to the instance and debug it. To do this, enter a command of the following form on the application server. 70 70 71 -{{ noformat}}70 +{{code}} 72 72 73 73 fooserver$ jdb -attach 8121 74 74 75 -{{/ noformat}}74 +{{/code}} 76 76 77 77 If you want to debug a remote machine you can use the following command. 78 78 79 -{{ noformat}}78 +{{code}} 80 80 81 81 foodev$ jdb -attach woserverhost:8121 82 82 83 -{{/ noformat}}82 +{{/code}} 84 84 85 -This may be useful in situations where your WebObjects application server does not have the ##jdb## tool installed and so you need to run the jdb tool from a host where the ##jdb## tool is installed. 84 +This may be useful in situations where your WebObjects application server does not have the ##computer code|jdb## tool installed and so you need to run the jdb tool from a host where the ##computer code|jdb## tool is installed. 86 86 87 -You will now be using the java debugger. There are a slew of commands that can help you work with the debugged java system, but this article is just going to focus on getting the thread stack traces. Issue the command ##suspend## to freeze all of the threads so they can be dumped and then the command ##where all## in order to get all the stack traces of the threads. Finally when you wish to resume the threads again, issue the command ##resume##. You're advised to actually quit the ##jdb## environment as soon as you have the information you need. 86 +You will now be using the java debugger. There are a slew of commands that can help you work with the debugged java system, but this article is just going to focus on getting the thread stack traces. Issue the command ##computer code|suspend## to freeze all of the threads so they can be dumped and then the command ##computer code|where all## in order to get all the stack traces of the threads. Finally when you wish to resume the threads again, issue the command ##computer code|resume##. You're advised to actually quit the ##computer code|jdb## environment as soon as you have the information you need. 88 88 89 89 = What to Look For = 90 90 91 91 An example of a stack trace is shown below. You'll notice the java class name and source-code line number at the end of a particular entry in the stack. Here we can see that the thread called "WorkerThread103" has stuck trying to get a session from the session store. In this situation another thread will most likely have the session store locked and is not releasing the lock. 92 92 93 -{{ noformat}}92 +{{code}} 94 94 95 95 WorkerThread103: 95 +{panel} 96 96 [1] java.lang.Object.wait (native method) 97 97 [2] java.lang.Object.wait (Object.java:429) 98 98 [3] com.webobjects.appserver.WOSessionStore.checkOutSessionWithID (WOSessionStore.java:207) 99 99 [4] com.webobjects.appserver.WOApplication.restoreSessionWithID (WOApplication.java:1,546) 100 - [5] com.webobjects.appserver._private.WOComponentRequestHandler._dispatchWithPreparedApplication 101 -(WOComponentRequestHandler.java:314) 100 + [5] com.webobjects.appserver._private.WOComponentRequestHandler._dispatchWithPreparedApplication (WOComponentRequestHandler.java:314) 102 102 [6] com.webobjects.appserver._private.WOComponentRequestHandler._handleRequest (WOComponentRequestHandler.java:358) 103 103 [7] com.webobjects.appserver._private.WOComponentRequestHandler.handleRequest (WOComponentRequestHandler.java:432) 104 104 [8] com.webobjects.appserver.WOApplication.dispatchRequest (WOApplication.java:1,306) ... ... @@ -106,16 +106,18 @@ 106 106 [10] com.webobjects.appserver._private.WOWorkerThread.runOnce (WOWorkerThread.java:173) 107 107 [11] com.webobjects.appserver._private.WOWorkerThread.run (WOWorkerThread.java:254) 108 108 [12] java.lang.Thread.run (Thread.java:552) 109 - 108 +{panel} 110 110 WorkerThread101: 110 +{panel} 111 111 [1] java.net.PlainSocketImpl.accept (PlainSocketImpl.java:351) 112 112 [2] java.net.ServerSocket.implAccept (ServerSocket.java:448) 113 113 [3] java.net.ServerSocket.accept (ServerSocket.java:419) 114 114 [4] com.webobjects.appserver._private.WOWorkerThread.run (WOWorkerThread.java:238) 115 115 [5] java.lang.Thread.run (Thread.java:552) 116 +{panel} 116 116 ... 117 117 118 -{{/ noformat}}119 +{{/code}} 119 119 120 120 If you have a look at what the other threads are doing at the same time, it is hopefully possible to ascertain what area might be at fault. At the very least, one can figure out what part of the application is at fault. 121 121