GSoC End - Current Status

Hi all.

The GSoC 2013 is now ended, and this post will describe shortly the current status of the project.

Of course, this sort of projects has always a load of things which can be done, and I'll continue to work on it, after this great summer of code. I'm pretty happy about how things went... I worked a lot, of course, but I enjoyed this experience a lot!

 After this brief introduction, I can declare the status of my project as checked in.
Many things can still be made, and some minor tweaks which can improve user experience.

You can find:

• full Zest documentation here (API), and here (wiki);
• Zest code here (Official) and here (my repo, a fork with my latest changes);
• Zap Code here;
• Zap-extensions code here.

Minor Tweaks:

• Adding ExpressionLength Dialog;
• Adding a hover over to the IF node showing the full expression tree;
• Same as before for AND/OR nodes;
• Change some menu names.

Some Screenshots of the Project:

Conditional UI

Structured Expression UI

SurroundWith menu for Expressions

A complex Script using Loops and Structured Expression (with debug output).

for more screenshots, contact me, or simply check it out and try it ;)

Things Made:

• Created Structured Expressions inside Zest;
• Added support for Structured Expression in ZestConditional & ZestAssertion
• Added Loop to Zest as a new Statement;
• Added support for the runner which manages loops of different types;
• Added UI in ZAP for the changes above;
• Tests;
• Docs.

For more information, please do not hesitate to write a comment or to email in the dev-groups or to me directly.

Week 14 - close to the end: cleaning and debugging

In this phase I'm cleaning the code and I'm making some other test to find out any other bug.

This is a sample screenshot of the working project:

Intermediate Post - ZestComplexExpression UI

This post only to show the current implementation of the ComplexConditionals UI:

The script implemented is very simple, but it tests complex conditionals.

Simply it checks if the status code is 200 AND if the url request is equals to http://localhost:8080/bodgeit/login.jsp (exactly the request added  in the script).

Then, it checks if the response time is >10 ms OR <100 ms (which is obviously true).

Complex Expressions are really close to be finished!

week 13 - Complex Conditionals UI

This is the 13th week. This week I continued working on the Complex Conditional UI.

I have developed two UIs for adding Complex Conditionals:

• via Script Tree: It is now possible to build a new conditional with empty expression. The expression will be then built step by step by adding AND/OR nodes: Here some screenshots:

and this is the result:

As you can see, the "IF" node contains no child (no expression).

User can then add other nodes, such as Simple Conditional, AND or OR nodes:

This is an example:

• via a complex Conditional Dialog:

This is the screenshot:

What is still missing?

I have to add some lines to manage elimination of complex conditional nodes; 

Some lines to perform a check about where is it possible to insert which type of conditional node;

A method to add the complete subtree given by a complete structured expression (built with the complex conditional dialog) .

Week 12 - Complex Conditional UI

This week I completed and pushed the definitive code for ZestLoops.
Then I started working on ComplexConditional. I created a sample UI based on the prototype I made in my proposal:

There are still many things to do for Complex Conditional. The most interesting challenge is to parse the condition and create the RootExpression for the structured conditional.

This week I wrote a basic standard text representation for all the simple conditionals, and I wrote the regex to recognize them.

Week 11 - Loops UI, tests and refractoring

This is the 11th week.

This week I worked a lot on loops, making the edits required to the loop interface.

These changes forced me to refractor the previous code. Also some problem occurred.

The main problem is about the serialization of loops. The test I made for serialization passed because they consisted on the serialization of a single Loop. This kind of test was not good, because it didn't consider the loop inside a Script. In fact the serialization of a Script containing a Loop didn't work properly.

This problem is caused by the usage of generics and a bad design of the structure of loops.

This week I noticed this problem and fixed it making some changes at the design:

• The set of token is now declared in the subtype (LoopInteger, LoopFile & LoopString);
• The step is declared only for LoopInteger;
• The loop phase now considers the  index of the current token, and no more the value of the token.

After these changes I worked also at the Loop UI, changing the previous dialogs, here some snapshots:

This is the dialog for String Loops:

This is the dialog for Integer Loops:

This is the dialog for File (only fuzzers files)

Thes interfaces are much more easy.

As you can see, I inserted a new field: "Variable Name". This is a change I had to made to loops because it helps a lot in  the usage of the tokens iside the loops.

This week I planned to clear the code and make a, hopefully, definitive push for Loops. 

Week - 10 Graphic User Interface

This week I started working on the GUI for Zest.
I started with the UI for Loops, since they should be much more easier than the Complex Conditionals.

Here some simple snapshot (click on the img for zooming):

The images above show how user can create loops graphically (in the example, loops through values stored in a file).

After this simple realization, I started developing another feature: "Surround with..". This feature allow users to add some statement, select them, and wurround them with a Loop!

This week I've implemented these two features.

Two things are missing now:

1. to allow user load fuzzer files for loops;
2. Complex conditionals UI.

Week 9 - ZestLoop development & Pull Request

This week I continued on ZestLoop, as the realization required much more time I expected.

This required so much time, because of a wrong  interpretation of the requirements.
Differences in the code are:

• elimination of the ZestLoopToken;
• change of the inheritance;
• the creation of the ZestLoopFile;
• the creation of a different type of TokenSet for integers (to allow loops as follows: FOR i FROM 0 TO 1000);
• some other minor stuff.

After this merge, I'll start working on the UI for ZAP.

Week 8 - ZestLoops development and improvements

8th week over... This week I continued on development of ZestLoops. Unfortunately some problems with serialization/deserialization with gson forced me to edit the previous code, and to differentiate loops  between String and Integer values.
This change allows also a better representation of Loops based on Integers value (i.e. in the form "FOR i FROM 0 TO 10000000" ): in fact, the previous code consisted on the creation of a set of integers between 0 and 10000000 (westfulness).
The new design splits the representation of Loops based on Strings (which uses more or less the previous code) and the one based on Integers; this second type only persists the State of the loop, i.e. the start value, the current value and the final one (in the example above, at step 5, only the values 0,5 and 10000000 are persisted).

This representation was discarded in the previous Loop code, because it didn't allowed loops of the form "FOR token IN [1,2,3,5,6,4,8,9]". After a chat with the mentor, we decided to represent this last type of loop as a ZestLoop based on String instead of Integers. The main reason of this decision is the following: such loop type could be used only if an user wants to take the value and put it inside the request/response. For this reason it is reasonable to interpret these values as Strings.

This week I also performed some tests on the previous code. Tests seem to work and passed also after the changes made!

I hope to be able to push the definitive code in few days!

Week 7 - ZestComplexConditional & ZestLoop

7th week is over. First official commit done!
Here the code :)

ZestStructuredConditional manages Structured conditionals.

How does it work?

The main idea was to create a load of expression types (based on REGEX, status code, URL, ...). Each ZestElement which needs to evaluate a list of expression build a StructuredExpression (which is an AND/OR of a list of other expressions). The mechanism is very simple: once the StructuredExpression had been built, the evaluation calls the method evaluate for each sub-expression. In this way an Expression Tree is built, and the evaluation mechanism works recalling the evaluation for each leaf; each internal node, which is an AND/OR expression, collect all the results of the children and compute an AND/OR evaluation of the boolean received by the children; the root will then return the value of the whole expression.

All tests of this work can be found in the official repo.

ZestLoop

Next task is a bit more tricky. Now Zest does not allow to use loop... I started developing this feature creating a new class: ZestLoop. This class should represent a particular statement which is a container of other statements.
I created a first draft of design and I started implementing how it should work. All the code I'm writing can be found in my repo. 

By the way, the current implementation is not really good. The biggest problem I found is that this does not separate the runner from the code: the current realization must have a reference to the whole Script and to the Runner. Loops are still under design phase, so I plan to solve this issue next week and, maybe, to make another pull request. :)

Here a snapshot of the UML of the current implementation:

Week 6 - Editing Zest Complex Conditional

This is the sixth week: the exam session is about to finish.
This week I worked on the code I wrote last week following some comments and suggesions from the mentor.
Most important point were:

• Creation of a new class: ZestStructuredExpression;
• Modified inheritance of classes

the previous two points were to avoid code duplication for the ZestExpression[And,Or]. The current structure is the following:

• Developed the code of the method deepCopy for all the new classes.

Last week I didn't wrote this method.

• Made some structural changes (e.g. deleted the reference to the parent of a conditional, the name, ...).

Next week is reserved for tests and, maybe, I'll try to integrate it at the ZAP side.

Week 5 - ZAProxy & Zest Complex Conditional

During 5th week I found a bug I what I developed last week. I started trying to solve the issue, but after some time spent on that problem, the mentor and I decided to start developing on Zest side: Complex Conditional.

Ref: https://docs.google.com/document/d/1oAx26AU5gs4WKVXeI1pQF-sC-B9CiVXd7R3PDz7Sikw/edit#heading=h.xgiwfxe7dwpf

The structure I came up now is this:

• ZestConditional

• ZestBooleanAnd
• ZestBooleanOr
• ZestExpressionRegex (regex exp1)
• ZestExpressionRegex (regex exp2)
• ZestBooleanStatusCode (200)
• List <ZestStatement> ifStatements etc...
  Going more inside, I designed the following internal structure for each class/interface:

• abstract class ZestExpression extends ZestElement implements ZestConditionalElement:
• List<ZestConditionalElement> children;
• boolean not;
• ZestConditionalElement parent;
• String name;
• static int counter; // for the default name
• abstract boolean evaluate;

• class ZestConditional extends ZestStatement implements ZestContainer, ZestConditionalElement:
• List<ZestConditionalElement> children;
• List<ZestStatement> ifStatement;
• List<ZestStatement> elseStatement;
• class ZestBoolean[And,Or] extends ZestExpression implements ZestConditionalElement:
• ZestConditionalElement parent;
• List<ZestConditionalElement> children;
• boolean evaluate();
• interface ZestConditionalElement extends ZestContainer
• getIndex(); //return the index of the statement;
• getChildren();// returns the children;
• isLeaf();//true if it has no children;
• isRoot();// if it is the root of the Conditional Tree
• evaluate();// evaluate the whole condition.

The structure above had been implemented and the javadoc is done, and this a basic UML:

And here you can find my fork of the Zest repo (with the structure developed): https://github.com/Vankar/zest

Some tests has started (very basic and not checked in).

Note the assumption: 

2 Expression of the same class can have the same name!

Any comment, tip or advice on the Structure of the Complex Conditional is greatfully welcome :)

Week 4 - Filters and ProxyScript

This week I tried to manage exams and coding as well as I could... and I think I had nice results :D

The goals of the week are the following:

1. Deeper understanding of the code;
2. First snippets of coding which can be used:
1. Filters & Containers;
2. UI.

Going deeper into the details:

I understood and learned (more and less) how ZAProxy is built (restrictly to those classes I worked with). Above all I understood how modular is the architecture of the program, and I improved knowledge on those modules I needed.

Then I continued playing with the code, trying to make something useful for the project: I started both from Zest and ZAP side:

Zest side:

I improved the design and I created the main interfaces and abstract classes (and some implementations)  for the replacing of the current ZAP Filters.
A more detailed description can be found here

ZAP side:

I worked on the UI. First I mistook to understand the requirements creating this:

(at least I learned how to create voices on the right click menu using ZAP classes :D ).And I started creating a Dialog to manage the filters.

Then I replaced with the correct UI:

(new icons needed!!!)

Unfortunally the exam period won't finish until 20th of July, then I do hope to increase substantially my speed on coding!!

Week 3 - ZAP Filters & Zest Scripts

Because of a couple of exams this week, I had to slow down a bit.

This week I concentrated  on the design of somewhat which could replace current ZAP Filters with Zest Scripts.
The goals of this week were to design such interfaces and to give some feedback on the usage of Zest.

Both the design and the example of usage for Zest can be found at this link, while the developed code can be found on my repo (only few lines :( but I hope they are well documented and well designed :D ).

[UPDATE July 21st] After a chat with the mentor, we fully designed the complete add-on. I also started with the integration of the Zap Filter replacement.

Week 2 - "learn by play"

This is the second week of GSoC. I started the paradigm "learn by play"!

These are the progresses:

• Started playing with the code of both ZAP and Zest;
• Started the design phase about the replacing of current Filters.
• Started writing some sample extensions for ZAP (which substantially do nothing);
• Gained a better knowledge of Zest, with some lack of the language;
• Understood better the steps I have to follow for the integration.

Nothing useful had been implemented yet, but I played a lot with the current code, and:

• I improved a lot my knowledge. 
• Much more aware of next steps!
• I had fun ;)

First Week

This is my first week of GSoC, and we are still in the Community Bonding period.

In this week I started looking deep inside the code and, after a chat with the mentor, we figured out the first development phase.

Replacing ZAP filters with ZEST

Filters in ZAP are not very used because of their lack of flexibility. The idea is to replace them completely with zest.
The designing phase has started.

Goals achieved

About the Community Bonding Period:

• I started reading documentation;
• Chatted in the IRC channel;
• Chatted with the mentor for more details and tips for a boost in first stage;

Outside the Community Bonding Period:

• First usecase for zest found: filters;
• Postponed the complex conditionals development inside zest;
• Better knowledge of ZAP and Zest code.

Accepted project: Dynamically Configurable actions add on

Short description: A plugin for OWASP Zed Attack Proxy will be developed to let users create and run Mozilla ZEST scripts with an as easy as possible interface.

Additional info: https://www.owasp.org/index.php/GSoC2013_Ideas#OWASP_ZAP:_Dynamically_Configurable_actions

Introduction

ZAP provides various mechanisms which allow HTTP requests and responses to be changed dynamically. So (for example) a string in an HTTP request can automatically be changed to another string.

It also supports a scripting interface, which is very powerful but at the moment difficult to use.

This project would introduce something inbetween thess 2 options - a powerful way of defining (potentially) complex rules using a wizard based interface.

The challenge will be to make it as usable as possible while still providing a wide range of functionality.

Zest is a specialized scripting language developed by the Mozilla security team and is intended to be used in web orientated security tools. A Zest add-on for ZAP already exist, but it is in a very early stage, and let the user only create very simple scripts.

This proposal will guarantee the user to create longer and more complex Zest scripts, introducing new structures as Complex Conditionals, Asserts and an easy and intuitive GUI, providing a wide range of functionality. 

Project Goals:

After discussing the idea with the mentor, I had set the following goals:

• Create a full documentation (both wiki and javadoc) for the current implementation of Zest and his add-on for ZAP: 
• the current implementation of zest contains no javadoc, and I think that, even if the code is quite simple to read, a full documentation for future implementations and extensions is reccomended; 
• Improve Zest (Complex Conditionals, Assertions, ...) 
• Find some good integration points and create a standard and extensible interface that can be easily reused to add new integration points (maybe introducing hooks like ActionHooks for extensions); 
• Create some zest examples reguarding the integration points found; 
• Create an intuitive GUI for a simple creation of the zest scripts, keeping in mind that: 
• the creation must be as simple as possible; 
• the GUI must guarantee a wide range of funcionality to the zest script; 
• Create all the documentation needed for an user with no experience in vulnerabilities and attacks to let him create a zest script. 

All the work must observe these requirements:

• The code must be: 
• Clean and easy to follow; 
• Include a full set of unit tests; 
• Include good and full documentation. 

Work and implementation
In this section I will describe more in detail the work. Please look the timeline in order to see how I suppose to split the work and not look at the order of the points in this section.

Documentation of the existing code
First step is the creation of the whole documentation for zest. Then the wiki will be completed with the missing pages in the zest-core section.

Design (detailed specs) & creation of usecases
Second step consists in a designing phase, in order to write down a simple and extensible interface for the creation of zest scripts.

This designing phase must be combined with some examples of usage, that will be considered as possible integration points for the addOn. By the way, the idea is to guarantee the usage of the add on for the examples above, but must be more flexible, in order to let users create their own usecases (in the future other extensions and integrations has to be developed easily).

Some usecases are the following:
Suppose to have a wizard with 3 steps (each of wich with a different anti CSRF token), and user wants to test a parameter in the last step, and he has to pass through the first two steps;
Suppose that a WEB application detects an attack and logged the user out; the user will be able to detect this with a zest script and reauthenticate in a very easy way;
Suppose a user wants to edit manually the requests (or run a script) when a given condition happens: he will simply set the conditional in the zest script and put a break point;
Suppose a user wants to test a set of string which can cause injection: he will simply set up the zest script changing the requests in order to test the strings and set a break point when an injection occurs.


other scenarios will be added during this phase.


Implementation
This part includes the following stages:

1. Improve the existing zest code: 
1. construption of Complex Conditions, collecting simple conditions (regex, status code, response time ones). More details about the implementation of this feature can be found here: https://groups.google.com/forum/?fromgroups=#!topic/mozilla-zest/5Uhs67ogLkg; 
2. managing break points (suspension of the script until an event occurs); 
3. managing scripts (suspension of the zest script, run of another script specified by user, resume of the zest script); 
2. Creating/improving the ZAP add on. 

The gui
The current version of the zest add-on guarantees a very unstructured way to add the requests/responses to the zest script, but the creation of IF/THEN/ELSE statements and the configuration of the actions is quite difficult. By the way, the usage of a wizard to set up the action could be too structured for the user.

In the following line I'll show you some extracts of the final GUI.


First this is a snaphot of the current way perspective:





In the current perspective the user is able to add IF/THEN/ELSE clauses and to modify the parameters of the requests.


This is the current perspective for manipulating requests:





It is a bit difficult to use: now we have only two parameters, but suppose to have a much longer body. It is quite difficult to manage it.




This second image represents a simpler way to edit requests and responses: the user should only edit the “editable” strings in the TextField.


With a right click the user is allowed to add new fields, remove fields, the add-on will then create the complete body of the request.


This perspective gives also the user the opportunity to mark some parts of the editable fields. These marked parts can then be changed with some other values (for example a list of String given by the user through a text file) or can be stored into token in order to be used again later in the script.


In the following image, is showed the current way to add a conditional.





To put some action inside the IF clause, the user has to right click on the if, and decide to add a new action, or condition.


The current perspective does not allow users to create more complex conditions: the usage of AND, OR, NOT clauses are not allowed here.


The final GUI will let the user write conditionals using a textField as the following:





lets call "condRegex1", "condResponseTime" and "conditionStatusCode" simple conditional. All the simple conditionals can both be defined in the text field or be defined through name. Suppose for example we want to define condRegex on the body of the request as follow:

"\Q<script>alert(1);</script>\E"


The application will then associate univocally condRegex1 and "\Q<script>alert(1);</script>\E".


The user will see a label containing the full declaration of the simple condition putting the cursor on it, and he will be able to edit the condition with a double click on the name of the simple condition.


The application will then parse the text field creating the full Conditional with a BuilderPattern.


In this case, the creation of the simple criterias remain more or less the same of the current version.

You can see a better description with a graphical perspective: https://groups.google.com/forum/?fromgroups=#!topic/mozilla-zest/5Uhs67ogLkg

and here you can download a little demo of the front end of this part:

https://dl.dropboxusercontent.com/u/5100168/OWASP/prototype.jar

The Repository is at this link:

https://bitbucket.org/vankar/zest-complex-conditional

The case of the break points can be managed with a pop up which rise with the correct condition. The popup is structured into tabs, and lets the user make all the actions he needs to. The zest script resumes once user press apply or ok to the pop up. 

Timeline
The project has substantially these deliverables:

• design; 
• develop; 
• documentation; 
• test. 

The main idea is to loop between these 4 steps:

1. first the design a new usecase of zap defining an example of usage; 
2. develop hooks and integration; 
3. creation of the documentation about the code already developed; 
4. test. 

This timeline is very flexible, and allows to put other phases, for example a review one, ie after people have had a chance to play with the code.

About me


Short bio / overview of your background:


2011-current:

M.S., University of Padua. Currently pursuing the M.S. in Computer Engineering.


2008-2012:

B.S., University of Padua, 93/110. Bachelor in Information Engineering


M.S. exams:

Data Bases 26/30


Computer Networks 30/30 cum laude


Parallel Computing 30/30 cum laude


Data Structures And Algorithms 2 30/30


Strategic Enterprise Management 30/30


Current GPA:
29.2/30

 
Coding Skills

What platform do you use to code? Hardware specifications and operating system

I always work under Linux. I often change distribution, but now I've installed Linux Sabayon (gentoo based & preferred one) and Ubuntu.


Hardware: Laptop Dell XPS 17: Specifications:
CPU: intel i7-740
RAM: 4GB DDR3 1.333 GHz
HDD: 500GB


Did you ever code in C or C++/Perl/python/..., yes/no? what is your experience?I have good exeperience with Java and C++ (Java is the better known). I can also have experience with BASH, C, little of Python and some others.

Here a little description of the projects I collaborated for:

Bachelor Thesis
Title
eLaw: automatic management of notarial deeds

Supervisor
Professor Enoch Peserico and Engineer Federica Bogo

Keywords
eLaw, automatic, management, notarial deeds

Description
This work consists on the description of the productive and working model of a new software, whose aim is the automation of notarial deeds management. This document explains the usage of important informations retrieved in a notarial deed and describes how these informations are used in order to fill automatically some forms under the ministerial directives. The software here described is an OpenOffice extension which is supposed to be a valid alternative of the actual notarial softwares semplifying and automating some of the notary work. The pros of this work are the easy-to-use interface and the automatization. Reading this work you'll also see how this application can be easily used in other contexts.

Experiences

2011-current, University of Padua

"Web Quality Project"I'm involved in the Web Quality Research Team. Our goal is to perform a complete crawl of the italian web in order to test and compare different ranking algorithms. My role is to manage the crawl (hardware and software perspective) and to write an efficient extension of the Heritrix web crawler. Here i am making a large use of Java and Bash, I'm also using Spring framework to write Heritrix extension, tomcat and jsp.


2012,ERASMUS-LLP (IP) Course “Secure WEB Applications: Best Practices for Protection & Development” Glamorgan University, Pontypridd
Intensive Program on the topic of Web Security and the best practices for Protecting & Development.

2012, University of Padua
DNA Assembly
I worked in a DNA Assembly project for my Parallel Computation exam. My role was to create and to study the efficiency of the parallelization related to the extraction and the filtering of reads. Here i used C++, POSIX threads.


2012, University of Padua
Covering Firm Web Application
I collaborated on the creation of a web app lication for my Data Bases exam. The application had to manage orders, catalogue and warehouse for a covering organization. Here i used Java, OpenOffice UNO library, Castor, JDOM, XML and DTD.


2011, University of Padua
GPS for maemo
I collaborated on the creation of an application for a Nokia maemo based n900 for the Embedded System exam. The goal of this app is to get data from GPS sensors and to plot this data in a 2D map. Here i used C++, Qt and Maemo framework.

If you apply for a project on our ideas list, have you experience in the areas listed under "Desired knowledge"?
As you can read above, or in my web site, I have the experiences listed as Desired Knowledge.

Contact
Full Name: Alessandro Secco

University / current enrollment: University of Padua (Italy). M.S. in Computer Engineer.

Web site: http://www.dei.unipd.it/~seccoale/

mail: seccoale@gmail.com
References

https://www.owasp.org/index.php/GSoC_SAT
https://www.owasp.org/index.php/GSoC2013_Ideas#OWASP_ZAP:_Dynamically_Configurable_actions
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/guifre/3006
https://groups.google.com/forum/?fromgroups=#!topic/zaproxy-develop/TloQLHSJmdE
https://developer.mozilla.org/en-US/docs/Zest
https://code.google.com/p/zap-extensions/wiki/AddOn_Zest
https://github.com/mozilla/zest/wiki
https://groups.google.com/forum/?fromgroups=#!topic/mozilla-zest/5Uhs67ogLkg




Any advice or comments would be gratefully welcome.