I have an apache 2.2.15 front end trying to pass code to tomcat7 with mod_proxy to fire off two different webapps with different base paths. I think I'm close, but I'm still not getting a web page returned. I'm getting the 404 requested resource not available. I am sure hoping this is something simple, but after a week and a half of banging my head against this wall, I have become log blind. Posting in hopes someone can see what I can't, or can correct some fundamental misunderstanding I might be having about how this can work.
BMC Remedy arsys with a http://rmdy-mtr-p1.domain.com/arsys/ port 8080. webapp base is located at /opt/local/bmc/ARSystem/midtier/
http://rmd-mtr-p1.domain.com/kinetic/ with a base at: /opt/apache/tomcat7.0/webapps/kinetic/ port 8009.
httpd.conf:
PidFile run/httpd.pid
Timeout 5400
ProxyTimeout 5400
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15
Listen 80
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
Include conf.d/*.conf
User apache
Group apache
### Section 2: 'Main' server configuration
ServerAdmin root@localhost
UseCanonicalName Off
<Directory />
Options FollowSymLinks
AllowOverride None
order deny,allow
Allow from all
</Directory>
<Directory "/var/www/html">
Options Indexes
Options +Indexes
AllowOverride None
Order deny,allow
Allow from all
</Directory>
HostnameLookups Off
ErrorLog logs/error_log
LogLevel debug
<IfModule mod_proxy.c>
ProxyRequests On
<Proxy *>
Order deny,allow
Deny from all
Allow from .sandia.gov
</Proxy>
<VirtualHost *:80>
ServerName rmdy-mtr-p1.domain.com
ServerAlias rmdy-mtr-p1
ProxyRequests off
ProxyPreserveHost on
ProxyPass /arsys/ ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/ retry=1 acquire=3000 timeout=600 Keepalive=On
ProxyPassReverse /arsys/ ajp://localhost:80/opt/local/bmc/ARSystem/midtier/
ProxyPass /kinetic/ ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/ retry=1 acquire=3000 timeout=600 Keepalive=On
ProxyPassReverse /kinetic/ ajp://localhost:80/opt/apache/tomcat7.0/webapps/kinetic/
</VirtualHost>
server.xml file:
<?xml version="1.0" encoding="ISO-8859-1"?>
<Server port="8006" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener"/>
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"/>
<!-- Global JNDI resources -->
<GlobalNamingResources>
<!-- Test entry for demonstration purposes -->
<Environment name="simpleValue" type="java.lang.Integer" value="30"/>
<Resource auth="Container" description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
name="UserDatabase"
pathname="conf/tomcat-users.xml"
type="org.apache.catalina.UserDatabase"/>
</GlobalNamingResources>
<!-- Define the Tomcat Stand-Alone Service -->
<Service name="Catalina">
<Connector port="8080" maxHttpHeaderSize="65000" URIEncoding="UTF-8"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" protocol="AJP/1.3" redirectPort="8080" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="false" />
<!-- Define a non-SSL HTTP/1.1 Connector on port 8009 -->
<Connector port="8009" maxHttpHeaderSize="65000" URIEncoding="UTF-8"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" protocol="AJP/1.3" redirectPort="8009" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="false" />
In my catalina.out log I am seeing several different issues. I believe that the:
11 May 2015 09:17:44 - Connects to rmdy-ars-p1:2222 through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
Is because the connection between the midtier and the ars server hasn't been set up yet. The whole log since I restarted it this morning and tested yet again is:
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxHttpHeaderSize' to '65000' did not find a matching property.
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxSpareThreads' to '75' did not find a matching property.
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'disableUploadTimeout' to 'false' did not find a matching property.
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxHttpHeaderSize' to '65000' did not find a matching property.
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxSpareThreads' to '75' did not find a matching property.
May 11, 2015 8:51:57 AM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'disableUploadTimeout' to 'false' did not find a matching property.
May 11, 2015 8:51:58 AM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlNamespaceAware' to 'false' did not find a matching property.
May 11, 2015 8:51:58 AM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlValidation' to 'false' did not find a matching property.
May 11, 2015 8:51:58 AM org.apache.catalina.startup.SetContextPropertiesRule begin
WARNING: [SetContextPropertiesRule]{Context} Setting property 'deployOnStartup' to 'true' did not find a matching property.
-------------------------- Last shutdown state was :{STATE=VIEWSTAT_COMPLETED, CACHE_TIMESTAMP=1431027208192}
Validating cache.....
Valid cache is found in path: /opt/local/bmc/ARSystem/midtier/cache/1431027208192
11 May 2015 08:52:27 - Root WebApplicationContext: initialization started
11 May 2015 08:52:27 - Refreshing org.springframework.web.context.support.XmlWebApplicationContext@2f035cf2: display name [Root WebApplicationContext]; startup date [Mon May 11 08:52:27 MDT 2015]; root of context hierarchy
11 May 2015 08:52:27 - Loading XML bean definitions from class path resource [META-INF/applicationContext.xml]
11 May 2015 08:52:28 - Bean factory for application context [org.springframework.web.context.support.XmlWebApplicationContext@2f035cf2]: org.springframework.beans.factory.support.DefaultListableBeanFactory@27aaa0d4
11 May 2015 08:52:28 - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@27aaa0d4: defining beans [proxyTemplate,configuration,formCache,fieldmapCache,formfieldCache,goatfiel MapCache,activelinkCache,fieldgraphCache,htmlDataCache,jsDataCache,vfhtmlDataCache,vfjsDataCache,arUserPreferencesCache,arUserSearchesCache,skinDefCache,skinsCache,IFormServiceCacheAspect,IFormFieldServiceCacheAspect,IFieldMapServiceCacheAspect,IGoatFieldMapServiceCacheAspect,IActiveLinkServiceCacheAspect,IFieldGraphServiceCacheAspect,IDHTMLServiceCacheAspect,IARUserPreferencesServiceCacheAspect,IARUserSearchesServiceCacheAspect,ISkinServiceCacheAspect,ControlFieldAspect,TrimFieldAspect,PanelFieldAspect,DataFieldAspect,VertNavBarAspect,ViewInfoAspect,ViewFieldAspect,AttachmentFieldAspect,TableFieldAspect,AppListAspect,formService,fieldMapService,formFieldService,goatfieldFactory,goatfieldMapService,goatfieldService,emitterFactory,fieldgraphService,skinService,activelinkService,arUserPreferencesService,arUserSearchesService,dhtmlService,dhtmlRequestService,com.remedy.arsys.goat.Form#0,com.remedy.arsys.goat.Form#1,com.remedy.arsys.goat.field.GoatField#0,com.remedy.arsys.goat.field.FieldGraph#0,com.remedy.arsys.goat.ActiveLink#0,com.remedy.arsys.stubs.GoatServlet#0,com.remedy.arsys.stubs.GoatServlet#1]; root of factory hierarchy
11 May 2015 08:52:28 - Trying to load configuration arsys_api.xml
11 May 2015 08:52:28 - Could not load optional configuration arsys_api.xml
11 May 2015 08:52:28 - Trying to load configuration default.xml
11 May 2015 08:52:28 - jrpcMode: true
11 May 2015 08:52:28 - jniLoadMode: 1
11 May 2015 08:52:28 - apiRecordingMode: 0
11 May 2015 08:52:28 - apiRecordingLogDir:
11 May 2015 08:52:28 - useConnectionPooling: true
11 May 2015 08:52:28 - maxProxiesPerServer: 80
11 May 2015 08:52:28 - idleConnectionsPerServer: 5
11 May 2015 08:52:28 - connectionTimeout: 0
11 May 2015 08:52:28 - connectionLifespan: 0
11 May 2015 08:52:28 - timeUnit: MINUTES
11 May 2015 08:52:28 - minimumSupportedServerRpcVersion: 12
11 May 2015 08:52:28 - connectionMaxRetries: 1
11 May 2015 08:52:28 - timeLagBetweenRetriesMillSec: 100
11 May 2015 08:52:28 - stringizeFieldAssignments: false
11 May 2015 08:52:28 - stringizeSetIfQualification: false
11 May 2015 08:52:28 - useLegacyQualParser: false
11 May 2015 08:52:28 - useLegacyAssignParser: false
11 May 2015 08:52:28 - useLegacyQualFormatter: false
11 May 2015 08:52:28 - useLegacyAssignFormatter: false
11 May 2015 08:52:28 - UsePooling: true
11 May 2015 08:52:28 - , Pool setting info: , # maxProxyPerServer- 80, # idleConnectionsPerServer- 5, connectionTimeout setting- 00:00:00, connectionLifespan setting- 00:00:00
11 May 2015 08:52:28 - Currently, connection pooling true
11 May 2015 08:52:28 - Client adjusts connection variables maximumPoolSize = 80 minIdleProxySize = 5 connectionTimeout = 0 timeUnit = null
11 May 2015 08:52:28 - Client sets useConnectionPooling to : true
11 May 2015 08:52:29 - Root WebApplicationContext: initialization completed in 1338 ms
11 May 2015 08:52:29 - Currently, connection pooling true
11 May 2015 08:52:29 - Client sets connection limits per server to :80
11 May 2015 08:52:29 - UsePooling: true
11 May 2015 08:52:29 - , Pool setting info: , # maxProxyPerServer- 80, # idleConnectionsPerServer- 5, connectionTimeout setting- 00:00:00, connectionLifespan setting- 00:00:00
11 May 2015 08:52:29 - Client sets useConnectionPooling to : true
11 May 2015 08:52:29 - APITIMEOUT from system property : null
11 May 2015 08:52:29 - APITIMEOUT from Environment : null
11 May 2015 08:52:29 - AROVERLAYGROUPS from system property : null
11 May 2015 08:52:29 - AROVERLAYGROUPS from Environment : null
11 May 2015 08:52:29 - fipsProviderJsafeJCE installed = false
11 May 2015 08:54:36 - Trying to load configuration arsys_api.xml
11 May 2015 08:54:36 - Could not load optional configuration arsys_api.xml
11 May 2015 08:54:36 - Trying to load configuration default.xml
11 May 2015 08:54:37 - Connects to rmdy-ars-p1:2222 through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
11 May 2015 08:54:37 - Running on Mid Tier version 8.1.02 201408260235
11 May 2015 08:56:43 - Connects to rmdy-ars-p1:2222 through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
11 May 2015 09:00:56 - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
May 11, 2015 9:01:08 AM com.sun.xml.ws.transport.http.DeploymentDescriptorParser parseAdapters
WARNING: WSSERVLET17: duplicate endpoint name
May 11, 2015 9:01:09 AM org.glassfish.gmbal.impl.JMXRegistrationManager resumeRegistration
2015-05-11 09:01:09,156 INFO [localhost-startStop-1 ] - Starting up Kinetic Task.
2015-05-11 09:01:09,167 INFO [localhost-startStop-1 ] - Preparing Kinetic Task Engine runtime.
2015-05-11 09:01:15,167 INFO [localhost-startStop-1 ] - Preparing the Remedy API.
2015-05-11 09:01:16,514 INFO [localhost-startStop-1 ] - Loading the Configuration Manager using data directory: /opt/apache/tomcat7.0/webapps/kineticTask/WEB-INF/data
2015-05-11 09:01:16,533 DEBUG [localhost-startStop-1 ] - Loading properties file: /opt/apache/tomcat7.0/webapps/kineticTask/WEB-INF/data/config.properties
2015-05-11 09:01:16,533 WARN [localhost-startStop-1 ] - The properties file does not exist: /opt/apache/tomcat7.0/webapps/kineticTask/WEB-INF/data/config.properties
2015-05-11 09:01:16,537 INFO [localhost-startStop-1 ] - This web application instance has not yet been configured.
Could not create log file null
Created default console logger
Skipping cache initialization in SurveyInit.
LoadError: no such file to load -- file:/opt/apache/tomcat7.0/webapps/kinetic/WEB-INF/lib/kinetic-cloner-lib.jar!/lib/kinetic_cloner/cloner
require at org/jruby/RubyKernel.java:1038
require at file:/opt/apache/tomcat7.0/webapps/kinetic/WEB-INF/lib/jruby-complete-1.6.5.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36
__file__ at lib/kinetic_cloner.rb:15
each at org/jruby/RubyArray.java:1612
(root) at lib/kinetic_cloner.rb:14
require at org/jruby/RubyKernel.java:1038
(root) at file:/opt/apache/tomcat7.0/webapps/kinetic/WEB-INF/lib/kinetic-cloner-lib.jar!/kinetic-cloner.rb:1
require at org/jruby/RubyKernel.java:1038
(root) at file:/opt/apache/tomcat7.0/webapps/kinetic/WEB-INF/lib/kinetic-cloner-lib.jar!/kinetic-cloner.rb:1
11 May 2015 09:03:01 - Connects to rmdy-ars-p1:2222 through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
ARERR [90] Cannot establish a network connection to the AR System server : Connection timed out rmdy-ars-p1:2222
at com.remedy.arsys.stubs.ServerLogin.<init>(ServerLogin.java:297)
at com.remedy.arsys.stubs.ServerLogin.get(ServerLogin.java:135)
at com.remedy.arsys.stubs.ServerLogin.getAdmin(ServerLogin.java:167)
at com.remedy.arsys.stubs.ServerLogin.getAdmin(ServerLogin.java:148)
at com.remedy.arsys.share.ServerInfo.get(ServerInfo.java:352)
at com.remedy.arsys.share.Cache$ServerReaper$1.run(Cache.java:850)
Caused by: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
at com.bmc.arsys.apitransport.ApiProxyJRpcBase.connectionTry(Unknown Source)
at com.bmc.arsys.api.ProxyJRpc.getRpcClient(Unknown Source)
at com.bmc.arsys.api.ProxyJRpc.<init>(Unknown Source)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at com.bmc.arsys.apitransport.connection.c.a(Unknown Source)
at com.bmc.arsys.apitransport.connection.c.createProxy(Unknown Source)
at com.bmc.arsys.api.ProxyManager.createProxy(Unknown Source)
at com.bmc.arsys.api.ProxyPool.createProxy(Unknown Source)
at com.bmc.arsys.apitransport.connection.a.get(Unknown Source)
at com.bmc.arsys.apitransport.connection.b.getProxy(Unknown Source)
at com.bmc.arsys.api.PoolingProxyManager.getProxy(Unknown Source)
at com.bmc.arsys.apitransport.connection.b.getProxy(Unknown Source)
at com.bmc.arsys.api.ARServerUser.usePrivateRpcQueue(Unknown Source)
at com.remedy.arsys.stubs.ServerLogin.<init>(ServerLogin.java:279)
... 5 more
11 May 2015 09:17:44 - Connects to rmdy-ars-p1:2222 through Java Rpc failed with: ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out rmdy-ars-p1:2222
I have tried a bunch of things. I tried doing this with mod_ajp first, because I'm more used to it. But it's better suited for situations like app1.company.com and app2.company.com and NOT company.com/app1 and company.com/app2. I could make any ONE work, but not two or three (my ultimate goal).
Speaking of situations, the third web app deployed in that webapps folder is kineticTask, and I don't know why it is firing off in the log files when I have no mention of it in my current config. I want to get these two working first and then add it....so I don't know why that's coming up at all.
The /var/log/httpd/error.log:
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(45): proxy: AJP: canonicalising URL //localhost:8080/opt/local/bmc/ARSystem/midtier/
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(1521): [client 134.253.176.154] proxy: ajp: found worker ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/ for ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/
[Mon May 11 10:24:00 2015] [debug] mod_proxy.c(1020): Running scheme ajp handler (attempt 0)
[Mon May 11 10:24:00 2015] [debug] mod_proxy_http.c(1957): proxy: HTTP: declining URL ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(709): proxy: AJP: serving URL ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(2026): proxy: AJP: has acquired connection for (localhost)
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(2082): proxy: connecting ajp://localhost:8080/opt/local/bmc/ARSystem/midtier/ to localhost:8080
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(2209): proxy: connected /opt/local/bmc/ARSystem/midtier/ to localhost:8080
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(2460): proxy: AJP: fam 10 socket created to connect to localhost
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(224): Into ajp_marshal_into_msgb
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[0] [Accept] = [text/html, application/xhtml+xml, */*]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[1] [Accept-Language] = [en-US]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[2] [User-Agent] = [Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[3] [Accept-Encoding] = [gzip, deflate]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[4] [Host] = [rmdy-mtr-p1.sandia.gov]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[5] [DNT] = [1]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[6] [Connection] = [Keep-Alive]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(450): ajp_marshal_into_msgb: Done
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(269): proxy: APR_BUCKET_IS_EOS
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(274): proxy: data to read (max 8186 at 4)
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(289): proxy: got 0 bytes of data
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 04
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(697): ajp_parse_type: got 04
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(516): ajp_unmarshal_response: status = 404
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(537): ajp_unmarshal_response: Number of headers is = 3
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[0] [Content-Type] = [text/html;charset=utf-8]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(609): ajp_unmarshal_response: ap_set_content_type done
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[1] [Content-Language] = [en]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[2] [Content-Length] = [1013]
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 03
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(697): ajp_parse_type: got 03
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 05
[Mon May 11 10:24:00 2015] [debug] ajp_header.c(697): ajp_parse_type: got 05
[Mon May 11 10:24:00 2015] [debug] mod_proxy_ajp.c(616): proxy: got response from (null) (localhost)
[Mon May 11 10:24:00 2015] [debug] proxy_util.c(2044): proxy: AJP: has released connection for (localhost)
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(45): proxy: AJP: canonicalising URL //localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(1521): [client 134.253.176.154] proxy: ajp: found worker ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/ for ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/
[Mon May 11 10:24:07 2015] [debug] mod_proxy.c(1020): Running scheme ajp handler (attempt 0)
[Mon May 11 10:24:07 2015] [debug] mod_proxy_http.c(1957): proxy: HTTP: declining URL ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(709): proxy: AJP: serving URL ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(2026): proxy: AJP: has acquired connection for (localhost)
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(2082): proxy: connecting ajp://localhost:8009/opt/apache/tomcat7.0/webapps/kinetic/ to localhost:8009
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(2209): proxy: connected /opt/apache/tomcat7.0/webapps/kinetic/ to localhost:8009
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(2460): proxy: AJP: fam 10 socket created to connect to localhost
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(224): Into ajp_marshal_into_msgb
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[0] [Accept] = [text/html, application/xhtml+xml, */*]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[1] [Accept-Language] = [en-US]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[2] [User-Agent] = [Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[3] [Accept-Encoding] = [gzip, deflate]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[4] [Host] = [rmdy-mtr-p1.sandia.gov]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[5] [DNT] = [1]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(290): ajp_marshal_into_msgb: Header[6] [Connection] = [Keep-Alive]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(450): ajp_marshal_into_msgb: Done
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(269): proxy: APR_BUCKET_IS_EOS
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(274): proxy: data to read (max 8186 at 4)
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(289): proxy: got 0 bytes of data
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 04
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(697): ajp_parse_type: got 04
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(516): ajp_unmarshal_response: status = 404
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(537): ajp_unmarshal_response: Number of headers is = 3
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[0] [Content-Type] = [text/html;charset=utf-8]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(609): ajp_unmarshal_response: ap_set_content_type done
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[1] [Content-Language] = [en]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[2] [Content-Length] = [1025]
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 03
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(697): ajp_parse_type: got 03
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 05
[Mon May 11 10:24:07 2015] [debug] ajp_header.c(697): ajp_parse_type: got 05
[Mon May 11 10:24:07 2015] [debug] mod_proxy_ajp.c(616): proxy: got response from (null) (localhost)
[Mon May 11 10:24:07 2015] [debug] proxy_util.c(2044): proxy: AJP: has released connection for (localhost)
So using Mod_jk I kept it simple. The httpd.conf file ONLY needs the redirect commands to call Tomcat to do all the work. Using the Virtual hosts entry only confuses the issue. Anything in the Virtual Host section Apache will work with. Anything with JKMount will go directly to Tomcat. So all httpd.conf needs to see is:
LoadModule jk_module /etc/httpd/modules/mod_jk_apache_2.2.Linux_64.so
JkWorkersFile /opt/apache/tomcat7.0/conf/worker.properties
JkLogFile /etc/httpd/logs/mod_jk.log
JkLogLevel info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
JkOptions +ForwardURIEscaped +ForwardURICompatUnparsed
JKMountCopy On
JkMount /arsys/* Worker1
JkMount /kinetic/* Worker2
JkMount /kineticTask/* Worker3
So if you hit webserver.com/arsys/ it hits httpd.conf, which sees this JKMount command. That redirects to the Worker1 property in Worker.properties which look like:
worker.list=Worker1
worker.Worker1.port=8009
worker.Worker1.host=localhost
worker.Worker1.type=ajp13
Keep in mind that EVERYTHING must be on a different port to avoid conflict so the other two are:
worker.list=Worker2
worker.Worker1.port=8019
worker.Worker1.host=localhost
worker.Worker1.type=ajp13
worker.list=Worker3
worker.Worker1.port=8099
worker.Worker1.host=localhost
worker.Worker1.type=ajp13
So that request is going to look here and see that Worker should be listening on port 8009 in Tomcat. Note: I initially thought that each Worker property needed to reflect the Worker# (1, 2, 3, etc...) I thought for example that worker.Worker1.port=8099 should be worker.Worker3, but it doesn't seem to matter.
So all you have to do in server.xml is open those ports. KISS...keep it simple:
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" URIEncoding="UTF-8" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
<!-- Define an AJP 1.3 Connector on port 8019 -->
<Connector port="8019" URIEncoding="UTF-8" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
<!-- Define an AJP 1.3 Connector on port 8099 -->
<Connector port="8099" URIEncoding="UTF-8" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
Tomcat will do the rest if there is a war file in its webapp folder.
The redirectPort threw me off. It will always stay "8443" even if you don't use SSL. It shouldn't loop itself, ie, Connector port="8099" redirectPort="8099" is bad.
That got me working.