hadoopmapreducejaxbguiceapache-crunch

Hadoop Job: Error injecting constructor, JAXBException


A MapReduce job implemented in an Apache Crunch pipeline is failing with the error message Error injecting constructor, javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported.

The Crunch pipeline is very similar to other functioning pipelines; does anyone have any theories or intuitions about the errors seen below?

Thank you for any help.

An excerpt of the application log can be found below:

2016-02-16 13:34:31,925 INFO [main] org.mortbay.log: Extract jar:file:/data7/hadoop/yarn/local/filecache/96/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar!/webapps/mapreduce to /data6/hadoop/yarn/local/usercache/anonymizedusername/appcache/application_1455519038503_6094/container_e26_1455519038503_6094_02_000001/tmp/Jetty_0_0_0_0_52016_mapreduce____.kjcl3z/webapp
2016-02-16 13:34:32,627 WARN [main] org.mortbay.log: failed guice: com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported
  at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver.<init>(JAXBContextResolver.java:70)
  at org.apache.hadoop.mapreduce.v2.app.webapp.AMWebApp.setup(AMWebApp.java:33)
  while locating org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver

1 error
2016-02-16 13:34:32,628 ERROR [main] org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@1981d861{/,jar:file:/data7/hadoop/yarn/local/filecache/96/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.1.2.3.4.0-3485.jar!/webapps/mapreduce}
com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported
  at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver.<init>(JAXBContextResolver.java:70)
  at org.apache.hadoop.mapreduce.v2.app.webapp.AMWebApp.setup(AMWebApp.java:33)
  while locating org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver

1 error
    at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
    at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
    at com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory$GuiceInstantiatedComponentProvider.getInstance(GuiceComponentProviderFactory.java:332)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory$ManagedSingleton.<init>(IoCProviderFactory.java:179)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory.wrap(IoCProviderFactory.java:100)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory._getComponentProvider(IoCProviderFactory.java:93)
    at com.sun.jersey.core.spi.component.ProviderFactory.getComponentProvider(ProviderFactory.java:153)
    at com.sun.jersey.core.spi.component.ProviderServices.getComponent(ProviderServices.java:251)
    at com.sun.jersey.core.spi.component.ProviderServices.getProviders(ProviderServices.java:148)
    at com.sun.jersey.core.spi.factory.ContextResolverFactory.init(ContextResolverFactory.java:83)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._initiate(WebApplicationImpl.java:1271)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.access$700(WebApplicationImpl.java:169)
    at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:775)
    at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:771)
    at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:771) 
    at com.sun.jersey.guice.spi.container.servlet.GuiceContainer.initiate(GuiceContainer.java:121)
    at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.initiate(ServletContainer.java:318)
    at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:609)
    at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210)
    at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:373)
    at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:710)
    at com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
    at com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
    at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
    at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
    at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
    at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
    at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
    at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
    at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
    at org.mortbay.jetty.Server.doStart(Server.java:224)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:857)
    at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:306)
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:142)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1148)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported
    at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:63)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:247)
    at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:234)
    at javax.xml.bind.ContextFinder.find(ContextFinder.java:441)
    at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:641)
    at com.sun.jersey.api.json.JSONJAXBContext.<init>(JSONJAXBContext.java:246)
    at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver.<init>(JAXBContextResolver.java:72)
    at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver$$FastClassByGuice$$d18e5b53.newInstance(<generated>)
    at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
    at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.Scopes$1$1.get(Scopes.java:65)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
    at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
    at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
    ... 50 more
2016-02-16 13:34:32,641 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:52016
2016-02-16 13:34:32,642 INFO [main] org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:52016
2016-02-16 13:34:32,642 WARN [2076864428@qtp-1188623367-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:52016] org.apache.hadoop.http.HttpServer2: HttpServer Acceptor: isRunning is false. Rechecking.
2016-02-16 13:34:32,643 WARN [2076864428@qtp-1188623367-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:52016] org.apache.hadoop.http.HttpServer2: HttpServer Acceptor: isRunning is false
2016-02-16 13:34:32,746 ERROR [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
    at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:310)
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:142)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1148)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.io.IOException: Unable to initialize WebAppContext
    at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:879)
    at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:306)
    ... 10 more
Caused by: com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting constructor, javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported
  at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver.<init>(JAXBContextResolver.java:70)
  at org.apache.hadoop.mapreduce.v2.app.webapp.AMWebApp.setup(AMWebApp.java:33)
  while locating org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver

1 error
    at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)
    at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
    at com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory$GuiceInstantiatedComponentProvider.getInstance(GuiceComponentProviderFactory.java:332)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory$ManagedSingleton.<init>(IoCProviderFactory.java:179)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory.wrap(IoCProviderFactory.java:100)
    at com.sun.jersey.core.spi.component.ioc.IoCProviderFactory._getComponentProvider(IoCProviderFactory.java:93)
    at com.sun.jersey.core.spi.component.ProviderFactory.getComponentProvider(ProviderFactory.java:153)
    at com.sun.jersey.core.spi.component.ProviderServices.getComponent(ProviderServices.java:251)
    at com.sun.jersey.core.spi.component.ProviderServices.getProviders(ProviderServices.java:148)
    at com.sun.jersey.core.spi.factory.ContextResolverFactory.init(ContextResolverFactory.java:83)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._initiate(WebApplicationImpl.java:1271)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.access$700(WebApplicationImpl.java:169)
    at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:775)
    at com.sun.jersey.server.impl.application.WebApplicationImpl$13.f(WebApplicationImpl.java:771)
    at com.sun.jersey.spi.inject.Errors.processWithErrors(Errors.java:193)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.initiate(WebApplicationImpl.java:771)
    at com.sun.jersey.guice.spi.container.servlet.GuiceContainer.initiate(GuiceContainer.java:121)
    at com.sun.jersey.spi.container.servlet.ServletContainer$InternalWebComponent.initiate(ServletContainer.java:318)
    at com.sun.jersey.spi.container.servlet.WebComponent.load(WebComponent.java:609)
    at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:210)
    at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:373)
    at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:710)
    at com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
    at com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
    at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
    at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
    at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
    at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
    at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
    at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
    at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
    at org.mortbay.jetty.Server.doStart(Server.java:224)
    at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
    at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:857)
    ... 11 more
Caused by: javax.xml.bind.JAXBException: property "retainReferenceToInfo" is not supported
    at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:63)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:247)
    at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:234)
    at javax.xml.bind.ContextFinder.find(ContextFinder.java:441)
    at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:641)
    at com.sun.jersey.api.json.JSONJAXBContext.<init>(JSONJAXBContext.java:246)
    at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver.<init>(JAXBContextResolver.java:72)
    at org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver$$FastClassByGuice$$d18e5b53.newInstance(<generated>)
    at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
    at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
    at com.google.inject.Scopes$1$1.get(Scopes.java:65)
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
    at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)
    at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
    at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)
    ... 50 more
2016-02-16 13:34:32,755 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2016-02-16 13:34:32,756 INFO [Socket Reader #1 for port 57690] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 57690
2016-02-16 13:34:32,762 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2016-02-16 13:34:32,762 INFO [IPC Server listener on 57690] org.apache.hadoop.ipc.Server: IPC Server listener on 57690: starting
2016-02-16 13:34:32,790 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2016-02-16 13:34:32,791 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2016-02-16 13:34:32,791 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2016-02-16 13:34:32,893 ERROR [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Exception while registering
java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:174)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:156)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:121)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:250)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:881)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1151)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
2016-02-16 13:34:32,893 INFO [main] org.apache.hadoop.service.AbstractService: Service RMCommunicator failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:177)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:121)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:250)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:881)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1151)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:174)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:156)
    ... 14 more
2016-02-16 13:34:32,895 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2016-02-16 13:34:32,895 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:177)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:121)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:250)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:881)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1151)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:174)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:156)
    ... 14 more
2016-02-16 13:34:32,896 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:177)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:121)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:250)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:881)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1151)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$5.run(MRAppMaster.java:1557)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1553)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1486)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:174)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:156)
    ... 14 more

Solution

  • It was a jetty dependency collision in the org.apache.avro:avro-mapred artifact.