Programming Thoughts
J2EE - Miscellaneous
Streaming Large Files in n-Tier

RMIIO

RMIIO is a Java library that solves this by sending data in small chunks over RMI. This is easy to use and server code can look like the following.

RemoteInputStreamServer streamServer = new SimpleRemoteInputStream(new FileInputStream(file), RemoteInputStreamServer.DUMMY_MONITOR, chunkSize); RemoteInputStream remoteInputStream = streamServer.export(); return new RemoteInputStreamDataVO(remoteInputStream, contentLength, mimeType);

Client code in a Struts 2 Action can look like the following, using IOUtils from Apache Commons IO.

response.setContentLength(streamData.getLength()); response.setContentType(streamData.getMimeType()); InputStream in = RemoteInputStreamClient.wrap(streamData.getRemoteInputStream()); OutputStream out = response.getOutputStream(); IOUtils.copyLarge(in, out);

This works a charm but, alas, this isn't a complete solution.

Users want random access

Consider an audio playback control like below. Users can skip back over time already played but can't skip forward any further than the buffered time.

Figure 1: Example audio playback control

This restriction doesn't occur when the web server serves the file directly. The difference is the Struts 2 Actions aren't responding with Accept-Ranges: bytes in the header, so the browser knows it can't request part of a file.

Random access from RMIIO

Trouble is, RMIIO doesn't grant random access out of the box. However, RMIIO can export an arbitrary input stream and an input stream can be concatenated from restricted streams of file byte ranges. These useful classes are Java SE SequenceInputStream and Apache Commons IO BoundedInputStream. Use code like the following.

/** * Returns input stream reading byte range requests from file. */ public static InputStream makeInputStreamFromByteRangeRequests(File file, List<ByteRangeRequest> byteRangeRequests) throws IOException { Vector<BoundedInputStream> byteRangeStreams = new Vector<>(); if (byteRangeRequests != null) { for (ByteRangeRequest byteRangeRequest: byteRangeRequests) { FileInputStream fileInputStream = new FileInputStream(file); fileInputStream.skip(byteRangeRequest.getStart()); BoundedInputStream boundedInputStream = new BoundedInputStream(fileInputStream, byteRangeRequest.getLength()); byteRangeStreams.add(boundedInputStream); } } return new SequenceInputStream(byteRangeStreams.elements()); } public static class ByteRangeRequest implements Serializable { private int start; private int end; public ByteRangeRequest(int start, int end) { this.start = start; this.end = end; } public int getStart() { return start; } public int getEnd() { return end; } public int getLength() { return end - start + 1; } }

Struts 2 Action

The following code is adapted from FileServlet supporting resume and caching and GZIP. First, read the byte ranges from the request header.

/** * Returns requested byte ranges from an HTTP 1.1 request. This fragment was adapted from BalusC's FileServlet * at http://balusc.blogspot.co.uk/2009/02/fileservlet-supporting-resume-and.html. * @throws IllegalArgumentException headerText does not conform to RFC 2616 byte range pattern or start is greater * than end. */ public static List<ByteRangeRequest> getByteRangeRequests(String rangeText, int contentLength) throws IllegalArgumentException { List<ByteRangeRequest> result; String numberText; int start, end; if (!rangeText.matches("^bytes=\\d*-\\d*(,\\d*-\\d*)*$")) { throw new IllegalArgumentException("HTTP Range header value is malformed range=" + rangeText); } result = new ArrayList<ByteRangeRequest>(); if (contentLength > 0) { // If file isn't missing for (String part : rangeText.substring(6).split(",")) { numberText = part.substring(0, part.indexOf("-")); start = (numberText.length() > 0) ? Integer.parseInt(numberText):-1; numberText = part.substring(part.indexOf("-") + 1, part.length()); end = (numberText.length() > 0) ? Integer.parseInt(numberText):-1; if (start == -1) { start = contentLength - end; end = contentLength - 1; } else if (end == -1 || end > contentLength - 1) { end = contentLength - 1; } if (start > end) { throw new IllegalArgumentException("HTTP Range header value is malformed range=" + rangeText); } result.add(new ByteRangeRequest(start, end)); } } return result; }

Next, is writing the response. Use the code below, which is the same as the article's except using the RMIIO stream rather than a file.

if (byteRangeRequests.size() == 1) { byteRangeRequest = byteRangeRequests.get(0); response.addHeader("Accept-Ranges", "bytes"); response.setContentType(partialStreamData.getMimeType()); response.setHeader("Content-Range", "bytes " + byteRangeRequest.getStart() + "-" + byteRangeRequest.getEnd() + "/" + contentLength); response.setHeader("Content-Length", String.valueOf(byteRangeRequest.getLength())); response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT); // HTTP 206. out = response.getOutputStream(); try { IOUtils.copyLarge(RemoteInputStreamClient.wrap(partialStreamData.getRemoteInputStream()), out); out.flush(); } finally { out.close(); } } else { response.setContentType("multipart/byteranges; boundary=MULTIPART_BYTERANGES"); response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT); // HTTP 206. servletOutputStream = (ServletOutputStream)response.getOutputStream(); // ServletOutputStream has handy println functions try { for (i = 0; i < byteRangeRequests.size(); i++) { byteRangeRequest = byteRangeRequests.get(i); servletOutputStream.println(); servletOutputStream.println("--MULTIPART_BYTERANGES"); servletOutputStream.println("Content-Type: " + partialStreamData.getMimeType()); servletOutputStream.println("Content-Range: bytes " + byteRangeRequest.getStart() + "-" + byteRangeRequest.getEnd() + "/" + contentLength); IOUtils.copyLarge(RemoteInputStreamClient.wrap(partialStreamData.getRemoteInputStream()), servletOutputStream, 0, byteRangeRequest.getLength()); } servletOutputStream.println(); servletOutputStream.println("--MULTIPART_BYTERANGES--"); servletOutputStream.flush(); } finally { servletOutputStream.close(); } }