Writing Filters for Apache 2.0
Subject:   Streaming of bulk data through a processing filter
Date:   2005-03-11 18:50:18
From:   nanyerri
Excuse my ignorance but I have not previously written Web services.

However, I am wanting to set up a Web service where client nominated files of indeterminate length (possibly > 10MBs) can be streamed into a data processing service. The service would process the data on the fly and stream the results back to the client.

The service is currently a Unix filter written in 'C' that performs sophisticated data manipulations on a record by record basis. At the moment, client's send their data electronically (FTP or CD), which is then processed in batch and returned.

The intent is to have the Web service act like a standard Unix filter that can be incorporated transparently into a client's own normal data processing logic.

For example:

<Client routines to extract data for processing> |
Call to Remote Web Data Processing Service |
<Client routines to store results of Web Service>

Is it possible with Apache filters (or any other Web Service) to support this type of processing? or, do the client's file and generated results have to continue to be FTP'ed between machines?