GithubHelp home page GithubHelp logo

icap-avscan's Introduction

Introduction

This project is intended to be used with Blue Coat ProxyAV (BCP), but other ICAP anti-virus system might work as well. The library is split into 2 versions, 1 written in Java and 1 written in C#. The versions are identical in behavior and more or less identical code-wise too.

The C# version has the addition of a 'FolderWatch' application (also available as a service) that watches for files added to a specified directory. When a new file is found, it is scanned and sorted depending on it's virus-status into 2 different subdirectories.

Please tell me, if any errors come up. I don't have access to any ICAP device anymore, so nothing gets fixed, unless it gets reported.

Example of use

Case 1

When a customer/user uploads a file on a web page it can be of any kind. This is a major security risk, if these files aren't virus-scanned. The FolderWatch subproject allows for a automatic scanning of these files and sorts them depending on the security risk. The FolderWatch will send the files individually to a BCP which checks the file and returns an answer. As an example, the result of the virus scan can be stored together with the account information of that customer/user to keep track of the files and their status.

The ICAP Protocol

Introduction

The Internet Content Adaption Protocol is heavily inspired by HTTP but the use differs on some core aspects. ICAP is normally implemented as an addition to HTTP, where the HTTP request for a web page can be encapsulated and modified before the user gets the content. This way, a content filter like a anti-virus software, can be transparent to the end-user. In this project, it is just used as a file transfer protocol with a feedback from the server about the file's virus-status.

Error codes

When working with the protocol, it often comes in handy to have the error codes and descriptions

Code Description
Informational Codes
100 Continue After ICAP preview
Success codes
200 OK
204 No modification needed
Client error codes
400 Bad request
403 Forbidden
404 ICAP Service not found
405 Method not allowed for service
408 Request timeout ICAP server gave up waiting for a request from an ICAP client.
Server error codes
500 Server error. Error on the ICAP server, such as ``out of disk space''.
501 Method not implemented. This response is illegal for an OPTIONS request since implementation of OPTIONS is mandatory.
502 Bad Gateway. This is an ICAP proxy and proxying produced an error.
503 Service overloaded. The ICAP server has exceeded a maximum connection limit associated with this service; the ICAP client should not exceed this limit in the future.
505 ICAP version not supported by server.

Use of The Libraries

Introduction

Both versions have the same interface of public and private methods. Although they are not equal in the way of handling exceptions. Both versions will disconnect automatically when caught by the garbage collection. The ICAP class is NOT thread-safe. If you have to transfer more than one file at a time, then instantiate the ICAP class for each connection.

Example of C# Version

The example below is fairly simple and involves only scanning 1 file. The scanFile() method returns true, if the file is clean and false if the file is infected.

class Program
{
    static void Main(string[] args)
    {
        try
        {
            // Starts a connection to the ICAP server
            ICAPNameSpace.ICAP icap = new ICAPNameSpace.ICAP("192.168.1.5",1344,"avscan");
            try
            {
                string file = @"C:\foo\bar.exe";
                Console.Write(file + ": ");

                // Sends the file for scanning and the result is stored in 'result'
                bool result = icap.scanFile(file);
                Console.WriteLine(result);
            }
            catch (Exception e)
            {
                Console.WriteLine("Could not scan file " + file + ":" + e);
            }
        }
        catch(Exception e)
        {
            Console.WriteLine("Error occurred when connecting" + e);
        }
    }
}

Example of Java Version

The example below is fairly simple and involves only scanning 1 file. The scanFile() method returns true, if the file is clean and false if the file is infected.

public class Tester
{
    public static void main(String[] args)
    {
        try
        {
            // Starts a connection to the ICAP server
            ICAP icap = new ICAP("192.168.1.5",1344,"avscan");
            try
            {
                String file = "\foo\bar.txt"
                System.out.print(file + ": ");

                // Sends the file for scanning and the result is stored in 'result'
                boolean result = icap.scanFile(file);
                System.out.println(result);
            }
            catch (Exception e) {
                System.out.println("Could not scan file " + file + ": " + e.getMessage());
            }
        }
        catch(Exception e){
            System.out.println("Error occurred when connecting" + e.getMessage());
        }
   }
}

Public Java methods

ICAP(String __IP__, int __port__, String __ICAP service__)

Given an IP-address, a port and an ICAP service name it will initialize a socket connection to the BCP. The preview size will be determined by an 'option' request from the BCP. This method throws an IOException if the socket connection or IO streams cannot be started. This lets the user of the class responsible for making a decision if such an error occurs. There is no good way to solve this exception automatically.

ICAP(String __IP__, int __port__, String __ICAP service__, int __preview size__)

Same as the one above, but the preview size assigned in the method call will be used to transfer files. Use this method to minimize overhead, but be sure to not change the BCP settings.

scanFile(String __filename__)

Given a filename, it will send the given file through the initialized connection to the BCP. If the file is clean it will return true and if the file is infected it will return false.

Public C# methods

ICAP(String __IP__, int __port__, String __ICAP service__, int __preview size (optional parameter)__)

Given an IP-address, a port, an ICAP service name and optionally an preview size it will initialize a socket connection to the BCP. If the preview size is assigned in the method call it will be used to transfer files. Use this method to minimize overhead, but be sure to not change the BCP settings. If the preview size is not assigned, the preview size will be determined by an 'option' request from the BCP.

scanFile(String __filename__)

Given a filename, it will send the given file through the initialized connection to the BCP. If the file is clean it will return true and if the file is infected it will return false.

Use of FolderWatch

Introduction

The FolderWatch subproject allow a Windows machine to continually scan files in a directory and sort them depending on their virus-status. FolderWatch can be run from a command prompt or as a service. The directory to scan and the subdirectories to sort files into are defined in the app.config file associated.

Setup

Navigate to the directory where the compiled FolderWatchService is located. Open the '.config' file and edit the appropriate settings (See Behavior).

Open a Visual Studio command prompt or a normal command prompt and note the location of your installutil.exe. Navigate to the directory where the compiled FolderWatchService is located. Now type either:

installutil FolderWatchService.exe

or if you are using a normal command prompt:

C:\The\Dirs\To\installutil.exe FolderWatchService.exe

Behavior

When the FolderWatch application starts, it will add all the current files in the directory to a scan-queue. This list is iterated through when there is open for new connections. The max amount of connections is specified in the app.config as 'maxInTransfer'. The scan-queue limit is defined in app.config as 'maxInQueue'.

Apart from adding all current files in the directory to the scan-queue, it will also watch the directory for created files. When a file is created, it is added to the scan-queue. The maxInQueue limit ensure that system memory is not clogged with waiting files. If the scan-queue is full when a new file is added, it will be ignored.

In the app.config a 'addAllFilesInterval' is defined in milliseconds. When the time has passed, it will added files in the directory to the scan-queue so that they aren't ignored completely.

The console application will print out to the command prompt with informations, error and so on. Alternatively, if it is installed as a service, it will send all messages to the system's event log.

icap-avscan's People

Contributors

baekalfen avatar gowtham-gopalakrishnan avatar marcschroeder avatar taoef1k avatar tlbdk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

icap-avscan's Issues

is getOptions call mandatory before the actual scanFile Call

Hi All,

This is more of a question rather than an issue. We have a use case where we are calling the ICAP Server from a Java Microservice. We are referring the Java Client code sample provided at https://github.com/Baekalfen/ICAP-avscan/tree/master/Java/ICAP/src/icap_samplecode

We observed that the getOptions call is taking a lot of time (around 10 secs minimum) and the actual scanFile Http call just takes milliseconds. Given we are not firing our requests from a browser and CORS is not in picture, can we skip the getOptions call?

Question: Why you get false on return code 200

Hello,
i have one question, why you in Java Client return false on return code 200 in the Line 176. The Snippets:
switch (status){
case 100: break; //Continue transfer
case 200: return false;
case 204: return true;
case 404: throw new ICAPException("404: ICAP Service not found");
default: throw new ICAPException("Server returned unknown status code:"+status);
}

Thanks a lot

McAfee Integration: Status Code 418

Hi,

When trying to implement with a McAfee server, I am recieving error code 418 which I believe means my request is missing a required request header.
I read the issue raised by sc06 and followed their advice (#8 (comment)) modifying the request as follows:

ESPMOD icap://<I.P.>/RESPMOD ICAP/1.0
Host: <I.P.>
X-Client-Abandon-Supported: 1
Preview: 30
X-Scan-Progress-Interval: 10
Allow: 204
Encapsulated: res-hdr=0, res-body=26

Content-Length: 572939

1e

I dont understand how sc06 created a request header.
Is the following from their comment the request header?:

"GET /C:\TestFiles\test1.txt HTTP/1.1
Host: icap.health.check"

Why would that need to be sent? Do I need to make the file available on that host?
Please forgive my lack of knowledge around the protocol.

Thanks very much for your help.

Charlie.

ICAPTester exception

I run the ICAPTester on my server but it thrrows an exception
Unhandled Exception: ICAPNameSpace.ICAP+ICAPException: Error in getHeader() method
at ICAPNameSpace.ICAP.getHeader(String terminator) in C:\Users\Dario\Desktop\ICAP-avscan-master\C-Sharp\ICAP\ICAP.cs:line 271
at ICAPNameSpace.ICAP.getOptions() in C:\Users\Dario\Desktop\ICAP-avscan-master\C-Sharp\ICAP\ICAP.cs:line 234
at ICAPNameSpace.ICAP..ctor(String serverIP, Int32 port, String icapService, Int32 previewSize) in C:\Users\Dario\Desktop\ICAP-avscan-master\C-Sharp\ICAP\ICAP.cs:line 68
at ConsoleApplication1.Program.Main(String[] args) in C:\Users\Dario\Desktop\ICAP-avscan-master\C-Sharp\ICAPTester\Program.cs:line 17

ICAP TIMEOUT

i don´t speak english

Hi, first thanks for share it
but i can´t run this, i send a simple text but doesn´t occur until the icap 408 error.

image

i attach wireshark captures.
image
image
image

Thanks !
Regars ;)

ICAP Java Example for encrypted communication

I was just wondering what must be changed in the example code to switch to an encrypted communication between the ICAP client and the server.

Serverside has configured the port 11344 to accept encrypted messages.

What else has to be changed beside the port to establish the connection?

I will be thankful for any hint.

Testing ICAP Solution

Hi,
I am sorry to post this question as an issue but, because here are the experts on this subject I thought I might get a good answer to my question.
I am trying to test my service that uses this code. I am able to send a EICAR test file through the service but ICAP Server is not considering it as "virus". The scan file method is always returning true(meaning the file is good). I am just trying to prove that my code can detect the virus file. Any input on testing this is appreciated. Thanks in Advance!

wrong bytes send as file data

In code of

while ((fileInStream.read(buffer)) != -1) {

we can see this lines:
while ((fileInStream.read(buffer)) != -1) {
sendString(Integer.toHexString(buffer.length) +"\r\n");
sendBytes(buffer);

But method InputStream.read() may not read full buffer. This method returns the total number of bytes read into the buffer, and this value should be used in next lines. For send string and as parameter for sendBytes() call.
In current state this method is sending additional bytes from buffer which were not filled by read() call.

License?

I see a LICENSE file in the Java section of the source tree but nothing in the csharp section. Could you clarify that the MIT license applies to the whole repo? Thanks much.

Macros

Does this detect virus in macros and scripts in MS documents?

Getting statuscode:418

I am using this ICAP client to scan the uploaded files. I receive the options as below--
ICAP/1.0 200 OK
Methods: REQMOD, RESPMOD
Options-TTL: ****
Encapsulated: null-body=0
Max-Connections: ***
Preview: 30
Service: McAfee Web Gateway 7.7.2 build 25114
ISTag: "00000000-00000000-00000000"
Allow: 204

Then when I try to send a file... I get 418 status code in response. the below is the request..
RESPMOD icap://.../RESPMOD ICAP/1.0
Host: ..
.
**
User-Agent: IT-Kartellet ICAP Client/1.1
Allow: 204
Preview: 30
Encapsulated: res-hdr=0, res-body=22

Content-Length: 52

1e

Could you please give me some pointers.. Appreciate you help! Thanks

A Poor performance on upload

There is a poor performance for upload (for Java version).
You write data (in sendString) directly into Socket's output stream without bufferization. It's a bad practice.
I've tested it on current implementation (without bufferization) a file (2Mb) had been uploaded in 1min, with bufferization it took 3sec.

Some implementations are missing?

I'm trying this library with Symantec Cloud services and for files that are clean, it's working fine. But for files that are infected, getting the following exception:

Could not scan file <filename>: Unrecognized or no status code in response header.

While digging deep, I found that if a file is infected Symantec will be returning 201 status code with the scan results.

For example, here's a sample response:

ICAP/1.0 201 Created
ISTag: "2E2FFEB048A3C546849453DD9DD868C2"
Date: Wed Aug 28 12:33:38 2019 GMT
Service: Symantec Protection Engine/8.0.0.48
Service-ID: Respmod AV Scan
X-Infection-Found: Type=0; Resolution=1; Threat=Trojan.Gen.2;
X-Violations-Found: 1
	7bed9479-e7c0-42b4-a3be-279cd0ece0f2-M-Mail1549367154632-ed057ecd-0d9f-4f08-85b2-f3454e58ed07-0.eml/Information.doc
	Trojan.Gen.2
	41129
	2
X-Outer-Container-Is-Mime: 1
Encapsulated: res-hdr=0, res-body=114

And it looks like we're missing some implementations here? (since it seems to be a standard as per https://onlinehelp.opswat.com/corev3/2._ICAP_response_headers.html). Not demanding, just want to know your viewpoint on this.

About thread blocking

when run sendString(requestBuffer) code in the scanFile method,I find that the thread is blocking.But i add thread.sleep(50) ,it run normally.What caused it? the code or ICAP server?

I am getting error in getHeaders method while sending \r\n\r\n as header ?

I am using the code as a rest web service and trying to upload the file as multipart .I have converted the file to an input stream and sent it to the scanfile method but when the debugger reaches to getHeader method I get in.read method result -1 and it throws ICAPException...any help will be appreciated.
Let me know if you need more details

having issue while scanning file greater than 2 mb

while((offset < STD_RECIEVE_LENGTH) && ((n = in.read(buffer, offset, 1)) != -1)) { // first part is to secure against DOS
			offset += n;
			//log.info("inside while");
			if (offset>endofheader.length+13){ // 13 is the smallest possible message "ICAP/1.0 XXX "
				//log.info("offset  "+offset);
				byte[] lastBytes = Arrays.copyOfRange(buffer, offset-endofheader.length, offset);
				log.info("lastBytes"+lastBytes.length);
				if (Arrays.equals(endofheader,lastBytes)){
					return new String(buffer,0,offset, StandardCharsetsUTF8);
				}
			}
		}
		throw new ICAPException("Error in getHeader() method");

if condition not stisfied
here i am getting last byte length 5 and it comes to exception block

java.net.SocketException: Broken pipe while scanning a file

I have c-icap installed and Try to scan a file using ICAP-avscan. I get the following error.

The below stack trace uses exactly your same class file ICAP, though only the package name is changed. Even VirusScanTest is exactly same as your file Test File.

java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:132)
at java.io.DataOutputStream.write(DataOutputStream.java:88)
at com.is.ICAP.sendBytes(ICAP.java:339)
at com.is.ICAP.scanFile(ICAP.java:156)
at com.is.VirusScanTest.main(VirusScanTest.java:18)

Please let me know if I am doing anything wrong.

Support ICAP DLP ?

hi ,
i am trying to connect the an icap server that scan files for dlp but not for AV scan.
its seems like everything is ok , the response look like ok .
but still we think that is not scan the file for dlp because the file should be bloke.
i want to know if the C# example here supporting an dlp Request ?
here is my Request:
byte[] requestBuffer = Encoding.ASCII.GetBytes(
"REQMOD icap://" + serverIP + "/" + icapService + " ICAP/" + VERSION + "\r\n"
+ "Host: " + serverIP + "\r\n"
+ "User-Agent: " + USERAGENT + "\r\n"
+ "Allow: 204\r\n"
+ "Preview: " + previewSize + "\r\n"
+ "Encapsulated: res-hdr=0, res-body=" + resBody.Length + "\r\n"
+ "\r\n"
+ resBody
+ previewSize.ToString("X") + "\r\n");
thanks for any help

java.nio.channels.SocketChannel instead of java.net.Socket for ICAP connection ?

Can we use java.nio.channels.SocketChannel, java.net.InetSocketAddress instead of java.net.Socket and import java.net.InetAddress for ICAP connection , so that we can make non-blocking IO connection and calls to send file to ICAP server for virus scanning.

Do you suggest any examples on how to make this work with java.nio.channels.SocketChannel and respective java.nio like java.nio.ByteBuffer rather than using java.io.DataOutputStream, BufferedReader and InputStreamReader etc., ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.