UberJar & CubaRestApiServlet not found

Support
#1

Could this issue https://github.com/cuba-platform/uberjar/issues/5 also be affecting my http->https handler in my jetty-conf.xml. In production we are running 6.10.8, in our dev environment we are on 7.1 and today I built my UberJar and released and now see the error below in app.log.

2019-10-28 19:53:04.682 WARN [main] org.eclipse.jetty.server.handler.ContextHandler.fins - unavailable

javax.servlet.UnavailableException: Class loading error for holder rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false

at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:99) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:367) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:749) ~[fins.jar:na]

at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]

at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[na:na]

at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.start(Server.java:418) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.doStart(Server.java:382) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at com.haulmont.uberjar.CubaJettyServer.start(CubaJettyServer.java:127) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.execute(ServerRunner.java:249) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.main(ServerRunner.java:39) ~[fins.jar:na]

2019-10-28 19:53:04.682 WARN [main] org.eclipse.jetty.util.component.AbstractLifeCycle - FAILED rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false: javax.servlet.UnavailableException: Class loading error for holder rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false

javax.servlet.UnavailableException: Class loading error for holder rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false

at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:99) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:367) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:749) ~[fins.jar:na]

at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]

at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[na:na]

at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.start(Server.java:418) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.doStart(Server.java:382) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at com.haulmont.uberjar.CubaJettyServer.start(CubaJettyServer.java:127) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.execute(ServerRunner.java:249) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.main(ServerRunner.java:39) ~[fins.jar:na]

2019-10-28 19:53:04.683 DEBUG [main] org.eclipse.jetty.servlet.ServletHandler - EXCEPTION

javax.servlet.UnavailableException: Class loading error for holder rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false

at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:99) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:367) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:749) ~[fins.jar:na]

at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]

at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[na:na]

at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.start(Server.java:418) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.doStart(Server.java:382) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at com.haulmont.uberjar.CubaJettyServer.start(CubaJettyServer.java:127) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.execute(ServerRunner.java:249) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.main(ServerRunner.java:39) ~[fins.jar:na]

2019-10-28 19:53:04.683 WARN [main] org.eclipse.jetty.webapp.WebAppContext - Failed startup of context o.e.j.w.WebAppContext@7d20d0b{/fins,jar:file:/home/mwhy@ad.gf.state.us/cuba/fish-surveys/uberJar/fins.jar!/LIB-INF/app,UNAVAILABLE}

javax.servlet.UnavailableException: Class loading error for holder rest_api@ebe7fb2f==com.haulmont.restapi.sys.CubaRestApiServlet,jsp=null,order=2,inst=false,async=false

at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:99) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:367) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:749) ~[fins.jar:na]

at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[na:na]

at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[na:na]

at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735) ~[na:na]

at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734) ~[na:na]

at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[na:na]

at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:369) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:854) ~[fins.jar:na]

at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:278) ~[fins.jar:na]

at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:119) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:167) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.start(Server.java:418) ~[fins.jar:na]

at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) ~[fins.jar:na]

at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) ~[fins.jar:na]

at org.eclipse.jetty.server.Server.doStart(Server.java:382) ~[fins.jar:na]

at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[fins.jar:na]

at com.haulmont.uberjar.CubaJettyServer.start(CubaJettyServer.java:127) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.execute(ServerRunner.java:249) ~[fins.jar:na]

at com.haulmont.uberjar.ServerRunner.main(ServerRunner.java:39) ~[fins.jar:na]
#2

Here’s my jetty-conf.xml.

<?xml version="1.0"?> 
<New id="httpConfig" class="org.eclipse.jetty.server.HttpConfiguration">
    <Set name="secureScheme">https</Set>
    <Set name="securePort">8443</Set>
</New>

<New id="httpsConfig" class="org.eclipse.jetty.server.HttpConfiguration">
    <Call name="addCustomizer">
        <Arg>
            <New class="org.eclipse.jetty.server.SecureRequestCustomizer" />
        </Arg>
    </Call>
</New>

<New id="connector" class="org.eclipse.jetty.server.ServerConnector">
    <Arg name="server">
        <Ref refid="Server" />
    </Arg>
    <Arg name="factories">
        <Array type="org.eclipse.jetty.server.ConnectionFactory">
            <Item>
                <New class="org.eclipse.jetty.server.HttpConnectionFactory">
                    <Arg name="config">
                        <Ref refid="httpConfig" />
                    </Arg>
                </New>
            </Item>
        </Array>
    </Arg>

    <Set name="port">8080</Set>
</New>

<New id="sslContextFactory" class="org.eclipse.jetty.util.ssl.SslContextFactory">
    <Set name="KeyStorePath">app/conf/keystore.jks</Set>
    <Set name="KeyStorePassword">rPD#Z5zK%rYYU26r09oI^0u!^</Set>
    <Set name="KeyManagerPassword">rPD#Z5zK%rYYU26r09oI^0u!^</Set>
    <Set name="TrustStorePath">app/conf/keystore.jks</Set>
    <Set name="TrustStorePassword">rPD#Z5zK%rYYU26r09oI^0u!^</Set>
</New>

<New id="sslConnectionFactory" class="org.eclipse.jetty.server.SslConnectionFactory">
    <Arg name="sslContextFactory">
        <Ref refid="sslContextFactory" />
    </Arg>
    <Arg name="next">http/1.1</Arg>
</New>

<New id="sslConnector" class="org.eclipse.jetty.server.ServerConnector">
    <Arg name="server">
        <Ref refid="Server" />
    </Arg>
    <Arg name="factories">
        <Array type="org.eclipse.jetty.server.ConnectionFactory">
            <Item>
                <Ref refid="sslConnectionFactory" />
            </Item>
            <Item>
                <New class="org.eclipse.jetty.server.HttpConnectionFactory">
                    <Arg name="config">
                        <Ref refid="httpsConfig" />
                    </Arg>
                </New>
            </Item>
        </Array>
    </Arg>

    <Set name="port">8443</Set>
</New>

<Call name="setConnectors">
    <Arg>
        <Array type="org.eclipse.jetty.server.ServerConnector">
            <Item>
                <Ref refid="connector" />
            </Item>
            <Item>
                <Ref refid="sslConnector" />
            </Item>
        </Array>
    </Arg>
</Call>
<!--<New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">-->
    <!--<Arg>-->
        <!--<New class="javax.management.remote.JMXServiceURL">-->
            <!--<Arg type="java.lang.String">rmi</Arg>-->
            <!--<Arg type="java.lang.String" />-->
            <!--<Arg type="java.lang.Integer"><SystemProperty name="jetty.jmxrmiport" default="7777"/></Arg>-->
            <!--<Arg type="java.lang.String">/jndi/rmi://<SystemProperty name="jetty.jmxrmihost" default="localhost"/>:<SystemProperty name="jetty.jmxrmiport" default="7777"/>/jmxrmi</Arg>-->
        <!--</New>-->
    <!--</Arg>-->
    <!--<Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>-->
    <!--<Call name="start" />-->
<!--</New>-->
#4

I should add that I am using Tomcat in development - MacOS/11.0.3+7 with CUBA Platform 7.1 - this configuration has been working now for a couple of weeks without issues.

I upgraded yesterday from java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64 to java-11-openjdk-11.0.5.10-0.el7_7.x86_64 on our RHEL test server.

The RHEL test server is where I receive this error.

I just attempted to use a Jetty configuration without the HTTP–>HTTPS handler and the same errors in app.log are thrown. Jetty config below.

<Configure id="Server" class="org.eclipse.jetty.server.Server"> <Call name="addConnector"> <Arg> <New class="org.eclipse.jetty.server.ServerConnector"> <Arg name="server"> <Ref refid="Server"/> </Arg> <Set name="port">8080</Set> </New> </Arg> </Call> <Call name="addConnector"> <Arg> <New class="org.eclipse.jetty.server.ServerConnector"> <Arg name="server"> <Ref refid="Server"/> </Arg> <Arg> <New class="org.eclipse.jetty.util.ssl.SslContextFactory"> <Set name="KeyStorePath">app/conf/keystore.jks</Set> <Set name="KeyStorePassword">mypassword</Set> <Set name="KeyManagerPassword">mypassword</Set> <Set name="TrustStorePath">app/conf/keystore.jks</Set> <Set name="TrustStorePassword">mypassword</Set> </New> </Arg> <Set name="port">8443</Set> </New> </Arg> </Call> </Configure>

#5

Hello @mwhy

You’ve migrated your 6.10.8 project up to 7.1 version, am I right?

Please check that web.xml in the web module does not contain any entries related to REST API Servlet.

Regards,
Daniil

#7

I have fixed the CubaRestApiServlet related errors by removing the REST Api related sections from web.xml.

However, the configuration I have in my jetty-conf.xml that handles the user entering HTTP and switching to HTTPS isn’t working anymore. This took me a long time to figure out and I am hoping that we can figure this out.

When I enter http://[serverurl]:8080/fins, where fins in my context name I get the error below. Entering https://[serverurl]:8443/fins works fine.

This automatic protocol switching is such a common requirement it should probably be in the CUBA Jetty docs and tested with each release.

HTTP ERROR 403

Problem accessing /fins/. Reason:

!Secure

Powered by Jetty:// 9.4.z-SNAPSHOT

I will keep on troubleshooting, but if you have any suggestions on how to setup Jetty to protocol switch from http to https please suggest them.

#8

Could you share sample project?

#9

I will try to next week, but the jetty-conf.xml at the top of this post is what I am using in production on RHEL with CUBA Platform 6.10.8 and this is working.

You should be able to take this and replace with your own SSL information in the sslContextFactory section.

I previously posted my research on setting up Jetty with forced HTTP–>HTTPS here Documentation: UberJAR - Force Jetty to redirect HTTP to HTTPS - CUBA.Platform, this should help setup a sample project too.