[FIXED] How to handle compressed (gzip) HTTP requests (NOT response) in JAVA servlet - Simple Example?

Issue

I struggled with this problem for quite some time; and after finding a simple solution... wanted to ask a question & answer!!

The question has been asked in different ways multiple times on stack overflow, and the accepted solutions are either partially correct and complex or talk about response compression.

Aggregating some old Q&A on this topic:

• wrong accepted ans/partially correct & complex subsequent ans.

  • HTTP request compression
  • How to decode Gzip compressed request body in Spring MVC: (Almost correct 2nd ans, but complex overkill)

• The accepted ans is wrong. It's about RESPONSE compression and not REQUEST.

  • Can I compress HTTP Requests using GZIP?
  • Compress an HTTP Request in java

• Similar Questions - terminated at "NO request compression"

  • Does Java HTTP Client handle compression

• A specific question & ans for Spring RestTemplate framework: How to zip- compress HTTP request with Spring RestTemplate?

Solution

A simple solution is by using a filter. (See servlet-filter tutorial)

Create a Servlet Filter:

• Make sure that the filter is called either first/before any filters which use request body.

I. Register filter in web.xml:

<filter>
    <filter-name>GzipRequestFilter</filter-name>
    <filter-class>com...pkg...GzipRequestFilter</filter-class>
</filter>
<filter-mapping>
    <filter-name>GzipRequestFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

---

II. Code for filter class:

public class GzipRequestFilter implements Filter {
    // Optional but recommended.
    private static final Set<String> METHODS_TO_IGNORE = ImmutableSet.of("GET", "OPTIONS", "HEAD");

    @Override
    public void doFilter(
            final ServletRequest request,
            final ServletResponse response,
            final FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;
        
        String method = httpServletRequest.getMethod().toUpperCase();
        String encoding = Strings.nullToEmpty(
            httpServletRequest.getHeader(HttpHeaders.CONTENT_ENCODING));

        if (METHODS_TO_IGNORE.contains(method) || !encoding.contains("application/gzip")) {
            chain.doFilter(request, response); // pass through
            return;
        }
        
        HttpServletRequestWrapper requestInflated = new GzippedInputStreamWrapper(httpServletRequest);
        chain.doFilter(requestInflated, response);
    }

    @Override
    public void init(final FilterConfig filterConfig) throws ServletException {}
    @Override
    public void destroy() {}
}

---

III. Followed by code for GzipInputStream wrapper:

// Simple Wrapper class to inflate body of a gzipped HttpServletRequest.
final class GzippedInputStreamWrapper extends HttpServletRequestWrapper {
    private GZIPInputStream inputStream;

    GzippedInputStreamWrapper(final HttpServletRequest request) throws IOException {
        super(request);
        inputStream = new GZIPInputStream(request.getInputStream());
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        return new ServletInputStream() {
            // NOTE: Later versions of javax.servlet library may require more overrides.
            public int read() throws IOException {
                return inputStream.read();
            }
            public void close() throws IOException {
                super.close();
                inputStream.close();
            }
        };
    }

    @Override
    public BufferedReader getReader() throws IOException {
        return new BufferedReader(new InputStreamReader(inputStream));
    }
}

---

Now what remains is how to send a compressed request?

Postman does not yet support sending compressed HttpRequest bodies. You can still make it work by using the binary option and use a gzipped file containing the properly encoded request body.

One way is using a nodejs script with pako compression library. For a multipart/form-data request see form-data library

const pako = require('pako')
const axios = require('axios')

var params = qs.stringify({
  'num': 42,
  'str': 'A string param',
});

data = pako.gzip(Buffer.from(params));

var config = {
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded',
    'Content-Encoding': 'application/gzip';
  },
}

await axios.post(
  'http://url-for-post-api-accepting-urlencoded',
  data,
  config,
).then((res) => {
  console.log(`status: ${res.status} | data: ${res.data}`)
}).catch((error) => {
  console.error(error)
})

---

NOTES:

• We are using Content-Encoding: application/gzip header to specify that a request is compressed. Yes this is standard.
• Do not use Content-Type as it will not work with multipart/form-data.
• The HTTP protocol has been running under the assumption that size of HttpRequests are dwarfed by HttpResponses.
• Further, due to assumed limited computing power in browser/client side, the norm has been to compress response and not requests. Browsers cannot natively compress but can do decompression natively.
• But, unfortunately after years of many developers pushing code; some HTTP APIs evolve to consume large strings/data!!
• It's a piece of cake to allow java servlets to have the option of working with compressed requests.

Answered By - dsculptor
Answer Checked By - Terry (JavaFixing Volunteer)