[FIXED] Spring Boot 2.1.6 can't start as a SystemD service

Issue

I'm trying to start a Spring Boot application as a Service on a Amazon Linux EC2 instance.

I'm packaging the service with:

$ mvn clean package spring-boot:repackage -Dstart-class=com.acme.MyMicroserviceApplication

When i start the service manually(without the service) it runs correclty:

$ /usr/bin/java -Djava.security.egd=file:/dev/./urandom -Dspring.profiles.active=pro -server -d64 -Xms256m -Xmx2048m -Xss256m -XX:+UseParallelGC -XX:MaxHeapSize=2048m -XX:MaxMetaspaceSize=1024m -XX:MaxHeapFreeRatio=75 -XX:+HeapDumpOnOutOfMemoryError -jar /home/ec2-user/jars/my-microservice-1.0.0-RELEASE.jar

INFO [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat started on port(s): 8080 (http) with context path '' INFO [main] org.springframework.boot.StartupInfoLogger: Started FacebookServiceApplication in 12.17 seconds (JVM running for 12.861) INFO [http-nio-8080-exec-1] org.apache.juli.logging.DirectJDKLog: Initializing Spring DispatcherServlet 'dispatcherServlet' INFO [http-nio-8080-exec-1] org.springframework.web.servlet.FrameworkServlet: Initializing Servlet 'dispatcherServlet' INFO [http-nio-8080-exec-1] org.springframework.web.servlet.FrameworkServlet: Completed initialization in 23 ms

But when i start the service, Spring Boot can not find the class:

• org.springframework.boot.loader.JarLauncher

The error is:

my-microservice[27205]: Error: Could not find or load main class org.springframework.boot.loader.JarLauncher
systemd[1]: squint-facebook.service: main process exited, code=exited, status=1/FAILURE
systemd[1]: Unit my-micro.service entered failed state.
systemd[1]: my-micro.service failed.
systemd[1]: Stopped my-microservice.

My service file is:

• /etc/systemd/system/my-micro.service

---

[Unit]
Description=my-microservice
After=syslog.target network.target
Wants=redis.service

[Service]
Type=simple
EnvironmentFile=/etc/sysconfig/my-microservice
Environment="JAVA_HOME=$(dirname $(dirname $(readlink $(readlink $(which java)))))"
Environment="PATH=$PATH:$JAVA_HOME/bin"
Environment="CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib"
Environment="SPRING_PROFILES_ACTIVE=pro"
Environment="JAVA_OPTS=-Djava.security.egd=file:/dev/./urandom -Dspring.profiles.active=pro -server -d64 -Xms256m -Xmx2048m -Xss256m -XX:+UseParallelGC -XX:MaxHeapSize=2048m -XX:MaxMetaspaceSize=1024m -XX:MaxHeapFreeRatio=75 -XX:+HeapDumpOnOutOfMemoryError"
User=springboot
# The configuration file application.properties should be here:
WorkingDirectory=/home/ec2-user/jars
# Run ExecStartPre with root-permissions
PermissionsStartOnly=true
ExecStartPre=/bin/mkdir -p /var/log/my-microservice
ExecStartPre=/bin/chown -R springboot:syslog /var/log/my-microservice
ExecStartPre=/bin/chmod -R 775 /var/log/my-microservice
ExecStart=/usr/bin/java \
        -Djava.security.egd=file:/dev/./urandom \
        -Dspring.profiles.active=pro \
        -server \
        -d64 \
        -Xms256m \
        -Xmx2048m \
        -Xss256m \
        -XX:+UseParallelGC \
        -XX:MaxHeapSize=2048m \
        -XX:MaxMetaspaceSize=1024m \
        -XX:MaxHeapFreeRatio=75 \
        -XX:+HeapDumpOnOutOfMemoryError \
        -jar my-microservice-1.0.0-RELEASE.jar
SuccessExitStatus=143
StandardOutput=journal
StandardError=journal
KillSignal=SIGINT
TimeoutStopSec=20
Restart=always
RestartSec=5
StartLimitInterval=0
StartLimitBurst=10
SyslogIdentifier=my-microservice

[Install]
WantedBy=multi-user.target

---

The pom.xml file of my project is:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.6.RELEASE</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.teamknowlogy.squint</groupId>
    <artifactId>my-microservice</artifactId>
    <version>1.0.0-RELEASE</version>
    <name>my-microservice</name>
    <description>...</description>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Greenwich.SR1</spring-cloud.version>
        <spring-session.version>1.3.5.RELEASE</spring-session.version>
        <maven.compiler.target>1.8</maven.compiler.target>
        <maven.compiler.source>1.8</maven.compiler.source>
        <!-- The main class to start by executing "java -jar" -->
        <start-class>com.acme.MyMicroserviceApplication</start-class>
        <spring.boot.mainclass>com.acme.MyMicroserviceApplication</spring.boot.mainclass>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-aws</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.session</groupId>
            <artifactId>spring-session-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.retry</groupId>
            <artifactId>spring-retry</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aspects</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-loader -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-loader</artifactId>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <artifactId>auth</artifactId>
            <groupId>software.amazon.awssdk</groupId>
        </dependency>
        <dependency>
            <artifactId>aws-core</artifactId>
            <groupId>software.amazon.awssdk</groupId>
        </dependency>
        <dependency>
            <artifactId>s3</artifactId>
            <groupId>software.amazon.awssdk</groupId>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>23.0</version>
        </dependency>
        <dependency>
            <groupId>net.sourceforge.htmlunit</groupId>
            <artifactId>htmlunit</artifactId>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt</artifactId>
            <version>0.9.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.12.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-pool2 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-pool2</artifactId>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <!-- https://mvnrepository.com/artifact/software.amazon.awssdk/bom -->
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>2.6.5</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <!-- https://spring.io/projects/spring-session -->
            <dependency>
                <groupId>org.springframework.session</groupId>
                <artifactId>spring-session-bom</artifactId>
                <version>Bean-SR3</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <resources>
            <resource>
                <directory>src/main/java</directory>
                <includes>
                    <include>**/*.properties</include>
                    <include>**/*.xml</include>
                </includes>
                <filtering>false</filtering>
            </resource>
            <resource>
                <directory>src/main/resources</directory>
            </resource>
        </resources>
        <!-- <sourceDirectory>src</sourceDirectory> -->
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <skipTests>true</skipTests>
                    <source>1.8</source>
                    <target>1.8</target>
                    <executable>true</executable>
                    <mainClass>${spring.boot.mainclass}</mainClass>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                        <configuration>
                            <mainClass>${spring.boot.mainclass}</mainClass>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

---

Version of Java:

$ /usr/bin/java -version
openjdk version "1.8.0_201"
OpenJDK Runtime Environment (build 1.8.0_201-b09)
OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)

Version of the Amazon Linux instance:

$ cat /etc/*release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
Amazon Linux release 2 (Karoo)

Solution

You use shell parameter expansion ($) in your Unit file Environment configuraton:

EnvironmentFile=/etc/sysconfig/my-microservice
Environment="JAVA_HOME=$(dirname $(dirname $(readlink $(readlink $(which java)))))"
Environment="PATH=$PATH:$JAVA_HOME/bin"
Environment="CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib"
Environment="SPRING_PROFILES_ACTIVE=pro"
Environment="JAVA_OPTS=-Djava.security.egd=file:/dev/./urandom -Dspring.profiles.active=pro -server -d64 -Xms256m -Xmx2048m -Xss256m -XX:+UseParallelGC -XX:MaxHeapSize=2048m -XX:MaxMetaspaceSize=1024m -XX:MaxHeapFreeRatio=75 -XX:+HeapDumpOnOutOfMemoryError"

Systemd doesn't support shell parameter expansion:

Environment= Sets environment variables for executed processes. Takes a space-separated list of variable assignments. This option may be specified more than once, in which case all listed variables will be set. If the same variable is set twice, the later setting will override the earlier setting. If the empty string is assigned to this option, the list of environment variables is reset, all prior assignments have no effect. Variable expansion is not performed inside the strings, however, specifier expansion is possible. The $ character has no special meaning. If you need to assign a value containing spaces or the equals sign to a variable, use double quotes (") for the assignment.

I would suggest to define the environment variables in a usual setenv.sh file of the application.

Answered By - Jürgen Hötzel
Answer Checked By - Terry (JavaFixing Volunteer)