Related pages
Bloggers
Tags
News
2009. 05. 21. 11:11:00.
Last week we hold a technical partners day. I was talking about logging. I put it here, hopefully somebody is interested;-)
What can we do if we want to know what is going through on our firewall? It fully depends on the type of our firewall. A well configured packet filter not only logs the denied traffic, but the allowed too. (I have already seen such a firewall which has not logged any traffic.) Let's see an example log of a netfilter/iptables. When the connection is started we will see this:
How can we do that with Zorp and PFservice? (PFService is for forwarding with kzorp and Zorp policy. So they use the same policy, therefore we do not need to administer twice). When the connection starts we will see the following in the logs:
If we are using Zorp Services (real proxies), on the default log level we can see the following. The client side connection started:
Some proxies can give addition useful information eg. HttpProxy about the URI:
What can we do if we want to know what is going through on our firewall? It fully depends on the type of our firewall. A well configured packet filter not only logs the denied traffic, but the allowed too. (I have already seen such a firewall which has not logged any traffic.) Let's see an example log of a netfilter/iptables. When the connection is started we will see this:
FORWARD: IN=br0 OUT=eth1 PHYSIN=eth0 SRC=192.168.1.1 DST=192.168.2.1 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=1398 DF PROTO=TCP SPT=1055 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0At the end we have almost the same message:
FORWARD: IN=br0 OUT=eth1 PHYSIN=eth0 SRC=192.168.1.1 DST=192.168.2.1 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=1389 DF PROTO=TCP SPT=1055 DPT=80 WINDOW=65535 RES=0x00 ACK FIN URGP=0How can we know that they belong to the same connection? From the following informations:
- Same source Ip (SRC) and port (SPT)
- Same destination IP (DST) and port (DPT)
- Same transport protocol (PROTO=TCP)
- The first log means the initiation (TCP SYN flag is on)
- The second one means the end (TCP FIN flag is on)
How can we do that with Zorp and PFservice? (PFService is for forwarding with kzorp and Zorp policy. So they use the same policy, therefore we do not need to administer twice). When the connection starts we will see the following in the logs:
kzorp (svc/intra_HTTPPF_internet:1): Forwarded session started; client_address='192.168.1.3:50587', client_zone='intranet', server_address='192.168.2.3:80', server_zone='internet', protocol='TCP'At the and:
kzorp (svc/intra_HTTPPF_internet:1) Forwarded session closedWhen we use UDP, the same message must come, but the protocol will be UDP;-)
If we are using Zorp Services (real proxies), on the default log level we can see the following. The client side connection started:
zorp/web[5086]: core.session(3): (svc/intranet_HTTP_internet:0): Starting proxy instance; client_fd='21', client_address='AF_INET(192.168.1.1:50834)', client_zone='Zone(intranet, 192.168.1.0/24)', client_local='AF_INET(192.168.2.200:80)', client_protocol='TCP'After that the server side connections start too:
zorp/web[5086]: core.session(3): (svc/intranet_HTTP_internet:0/http): Server connection established; server_fd='24', server_address='AF_INET(192.168.2.200:80)', server_zone='Zone(internet, 0.0.0.0/0)', server_local='AF_INET(192.168.2.254:39778)', server_protocol='TCP'What do theses parameters mean?
- core.session(3): the log class and the level
- client_address: the client IP address
- client_local: the IP were the client wants to go
- client_zone: the client's zone
- server_address: the IP where the Zorp connects
- server_local: the IP, what Zorp uses as a source IP
- server_zone: The server's zone
- intranet_HTTP_internet:0: the started service ID and the number of the session
- zorp/web: the proces name (zorp) and the instance ID
zorp/web[5086]: core.session(4): (svc/intranet_HTTP_internet:0): Ending proxy instance;Okay. Now we know the wheres and whens. It would be good to see the whos. All in all, traditional firewalls make their decisions (DAC: Discretionary Access Control) upon the client IP and server IP and the target ports. In addition proxies also checks if the parties keep the protocol (really HTTP is going on port 80?) According to the available logs we have now we do not have exact information who has made the connection, because:
- From one IP multiple users can come
- The same user can use different machines
- Maybe someone used the machine without the others knowledge
- Because of a possible malicious application the the PC
- Some applications connects to the vendor's website (for searching new versions) when the started. Maybe it is a useful information for the vendor that we do not want to tell.
zorp/web[5358]: core.auth(3): (svc/intranet_HTTP_internet:0/http): User authentication successful; entity='balabit', auth_info='inband'In the auth_info 'inband' means we used the browser as an auth client. The entity holds the UID. After the session id '/http' indicates the proxy class. The next messages is a failed authentication:
zorp/web[5584]: core.error(2): (svc/intranet_HTTP_internet:1/http): Error occured during authentication, credential is not accepted; method='REJECT'Finally we can see som extra info about the amount of data and duration of the connection. To enable it, just enable accouning logs (--log-spec '*.accounting:4'):
zorp/web[5358]: core.accounting(4): (svc/intranet_HTTP_internet:0/http/client): accounting info; type='ZStreamFD', duration='10', sent='1825',Of course we have different logs for the client and the server side. In the session id '/client' shows the client side traffic, while '/server' means the opposite. The downloaded amount of data is the received and uploaded is indicated by sent. The time of the connection is indicated by 'duration'.
received='445'
zorp/web[5358]: core.accounting(4): (svc/intranet_HTTP_internet:0/http/server): accounting info; type='ZStreamFD', duration='10', sent='402',
received='1825'
Some proxies can give addition useful information eg. HttpProxy about the URI:
zorp/web[5358]: http.accounting(4): (svc/intranet_HTTP_internet:0/http): Accounting; command='GET', url='http://www.balabit.com/'No we have complete picture about what is happening on our borders. I hope I coud give you a more detailed info about Zorp's logging capabilities and it is appetizing enough for ZAS testing. Thank you for the attention.
Posted by 
Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 03. 08. 11:13
It was in November 2007 when the initial commit of X Control Box was pushed to our VC server -- the idea was to create a framework based on the current Shell Control Box codebase that can act as a base for all our future appliance-like products. We took the chance to rewrite everything that needed to be rewritten: we added proper widgeting for our interface elements, cleared up the CSS code and the browser-server communication, the search interfaces, the whole permission system and added detailed changelogging for everything that can be changed on the UI. The resulting codebased proved to be usable: it took only a couple of weeks to create the initial pre-alpha version of syslog-ng Store Box, and ever since then we've really benefited from having a well-usable common framework into which 2630 commits have added 300k+ lines of code and templates by now.
The video below shows this progress from the beginning to the current state using the amazing visualization tool Gource. Make sure you click over to YouTube and watch it in high quality, it really worth it!
The video below shows this progress from the beginning to the current state using the amazing visualization tool Gource. Make sure you click over to YouTube and watch it in high quality, it really worth it!
Posted by Péter Gyöngyösi, Senior Developer
Péter Gyöngyösi, Senior Developer2010. 03. 06. 15:45
Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called "afsocket".
The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory.
By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL.
And the same plugin framework will enable me to create a wide variety of plugins. My ideas:
Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :)
The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory.
By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL.
And the same plugin framework will enable me to create a wide variety of plugins. My ideas:
- Plugins for all syslog-ng components (source, destination, filter, rewrite, parser)
- Python scriptability (a simple correllation engine in Python?)
- macro transformation functions, for example: $(stripslashes $macro), usable anywhere in templates and stripslashes a plugin that is invoked whenever such an expansion occurs
- Hooks for transforming the log message as it enters syslog-ng (to fix parsing errors for example),
Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :)
Posted by Balázs Scheidler, Chief Executive Officer
Balázs Scheidler, Chief Executive Officer2010. 03. 06. 07:32
Things have been a little rough last couple of months, that's why I haven't posted here. I'm in a rush right now as well, but I just wanted to let you know that I have started working on modularizing syslog-ng.
It is only a preliminary prototype, and as of now it doesn't compile, but the way it's going to work is already visible: each plugin will have its own plugin and with some trickery the large syslog-ng.conf parser will call out to the plugin parser. The user will recognize such a plugin as an integral part of syslog-ng.
E.g. this is a sample configuration file:
@version: 3.0
@module: dummy
...
destination d_dummy { dummy(dummy_opt(yes)); };
...
See the dummy plugin code in my git repository, in the "plugins" branch. Please note that that branch is going to be rebased a couple of times yet, I've released it in the spirit of "release early, release often".
I hope to get some of the recent contributions into plugins, instead of bloating the core syslog-ng code. For example output colorization. I'm also thinking about adding built-in scripting support via Python.
It is only a preliminary prototype, and as of now it doesn't compile, but the way it's going to work is already visible: each plugin will have its own plugin and with some trickery the large syslog-ng.conf parser will call out to the plugin parser. The user will recognize such a plugin as an integral part of syslog-ng.
E.g. this is a sample configuration file:
@version: 3.0
@module: dummy
...
destination d_dummy { dummy(dummy_opt(yes)); };
...
See the dummy plugin code in my git repository, in the "plugins" branch. Please note that that branch is going to be rebased a couple of times yet, I've released it in the spirit of "release early, release often".
I hope to get some of the recent contributions into plugins, instead of bloating the core syslog-ng code. For example output colorization. I'm also thinking about adding built-in scripting support via Python.
Posted by Balázs Scheidler, Chief Executive Officer
Balázs Scheidler, Chief Executive Officer2010. 03. 05. 14:26
I have just found a bug in syslog-ng OSE 3.1beta2. When you use keep_hostname(yes) in the options, then at those sources where this field is empty (e.g. messages arriving on /dev/log), it eliminates the hostname, somehow this way:
After the patch:
Not a big bug, but was very embrassing. The patch is in Bazsi's git tree out there. Thank you Bazsi!
I wish you a pleasant weekend!
myuser@myhost:~$ logger hello
myuser@myhost:~$ grep hello /var/log/syslog
myuser@myhost:~$ 2010-03-05T14:05:33+01:00 myuser[]: hello
After the patch:
myuser@myhost:~$ logger hello
myuser@myhost:~$ grep hello /var/log/syslog
myuser@myhost:~$ 2010-03-05T14:05:33+01:00 myhost myuser[]: hello
Not a big bug, but was very embrassing. The patch is in Bazsi's git tree out there. Thank you Bazsi!
I wish you a pleasant weekend!
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 03. 05. 13:58
It is well known that people and companies can get very innovative if they wish to cut the costs, and it is especially great if this results in a smart solution.
Making a market survey or buying an existing one is not cheap, and getting one for free can be a large benefit for smaller companies.
I felt handicapped from the start that as a small Hungarian company, we haven't had much information about the global market we were aiming at. Purchasing a global market study cost much more than what we could have afforded.
I was hoping that all this will change when we become a bigger company. And now, although we are still not that big, we could afford to buy a study. But there is always something that seems to be more important and promises imminent benefits. I have realized that we will not spend on that sort of thing until its price is higher than the margin of error in our marketing budgets.
But if everything goes well, this will not even be necessary, because our latest initiative offers free market data for everyone.
Communities have always had high priorities for BalaBit: they are part of our philosophy, our strategy and everyday life. We have always been an active member of the open source software community, and our products are based open formats and open standards.
The Open IT Survey 2010 project offers nothing less than a market survey to be filled by community members who are free to use the results. Participants can complete the anonymous questionnaire in a few minutes, and gain access to global marketing data about IT market trends.
The members of the community have the needed knowledge, we should just collect and share it for everyone's benefit. The open-source movement owes its success to the community, and we are positive that communities are important in shaping the future.
The Open IT Survey wishes to collect the last year's results and this year's expectations, focusing on the performance and trends of the IT and IT security fields. The project has been launched only recently; the first results will be available at the beginning of April at the openITsurvey2010.balabit.com site. According to the organizers, the target group is "everyone who is interested in the results."
The project builds its communication heavily on the community; participants can recommend the project to others on the Facebook Community Portal, as well.
We would like the project to become a regular survey, and gradually collect and share more and more information to everyone, therefore we count on the community's feedback and opinion. Comments and ideas are welcome on the Facebook page.
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager2010. 03. 02. 17:10
I start a new series on Hungarian sights to inform you about our country. The first post is about the Cave Bath which is the only thermal cave in Europe. Some weeks ago I was there with my girlfriend and it was really impressive.
The Cave Bath is a thermal bath in a natural cave in Miskolctapolca, which is part of the city of Miskolc, Hungary, and is unique in Europe.
The thermal water (temperature: 30°C, 86°F) is reputed to cure pain in the joints, and since it has lower salt content than most thermal waters (around 1000 mg/litre), people can bathe in it for much longer, practically for an unlimited amount of time. The Cave Bath can be visited all year long except for January.
The cave and the thermal spring have been known since ancient times, but Tapolca became a popular bathing place only after the Ottoman occupation of Hungary (16th-17th century). During this time the area belonged to the Greek Orthodox abbey of Görömböly; the development of Tapolca into a bathing place was the idea of the abbot in 1711. He also brought doctors from Kassa, today Koąice, Slovakia to examine the beneficial effects of the water. Three pools and an inn were constructed in 1723. The cave itself was not used yet, as the pools were outside. The water was colder than it is now, because the cold water springs of Tapolca (now playing an important role in providing Miskolc with drinkable water) were used too. By the mid-18th century, after a short period of popularity, the bath was neglected and by the 1800s the buildings were in ruins.
In 1837 the new abbot of Görömböly had the buildings restored and expanded. He also had the first indoor pool (though still outside the cave) built, but only for wealthy guests.
In the early 20th century the growing city Miskolc bought the area from the Greek Orthodox Church not only because of the thermal water but also because of the drinking water source (which now provides half of the city's water supply).
In the following years, new public baths were built. In 1934 Tapolca was officially recognised as a spa town. In 1939 the construction of a new bathing house began. During the construction several archaeological findings were unearthed, and a new, previously unknown water spring was found with a water temperature of 31.5°C. The thermal bath was opened in 1941 but the Cave Bath itself was opened only on 14 May 1959.
Since then the bath complex has been expanded several times. The outdoor pool and the characteristic, seashell-shaped roof before it were built in 1969. In the 1980s new rooms and corridors were built, and warmer pools (34°C and 36°C) were constructed. The newest expansion of the bath complex started in 1998.
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager2010. 02. 26. 14:21
I have finished the first online training. We used webex and phone conference. I must tell it was very tiring, but fun. Hopefully there will be more and more training whit this green way of teaching.
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 02. 24. 10:26
Let's imagine a webserver with many virtual host. It is jailed to a chroot, but it could be on a virtual machine (even a real one). I do want to collect the logs with the following requirements:
- No messages available by apache
- No need to handle hundres of sources. Transfer every log on one channel (okay let's use two;-)
- No need to rotate and support archive
- Use secure protocoll, but I guess I do not have to even mention;-)
mknod /var/log/apache/access.log pNow set all the virtualhost to use them:
mknod /var/log/apache/error.log p
ErrorLog /var/log/apache2/error.logImportant, that all the virtual hosts will use these nodes, therefor logs are not stored in the apache chroot any more. Now we have to read messages some way. The only small problem is apache uses Common Log Format, which is fare from any standard syslog format. Fortunateley it is possible to modify it in apache conf. The original looks this:
LogLevel debug
CustomLog /var/log/apache2/access.log combined
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedLet's change it to comply to syslog RFC mine looks this:
LogFormat "<123>Jan 12 12:12:12 %v apache[666]: %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedDo not care about strange PRI field and the fixed timestamp neither the funny PID. I only used it because google indexing;-) Which really count is hostname, which continas the name of the virtual host (%v = virutal host) and the original combined message is delivered at the MESSAGE field ($MSGONLY). So now we are ready with the apache side, let's focus on the syslog-ng side. The first step is reading the two pipes:
source s_apache_access {On the destination (writin side) we simple sore in different files by hostname field:
pipe("/var/log/apache2/access.log);
};
destination d_apache_access {I like solving archiving under the same time, therefore the second destination does it. Now I have to wire the client and the server side:
file("/var/log/apache2/$FULLHOST" template("$MSGONLYn") template-escape(no) owner("root") group("adm") perm(0640));
file("/var/log/archive/$R_YEAR/apache/$R_MONTH/$FULLHOST.$R_DAY" template("$MSGONLYn") template-escape(no) owner("root") group("adm") perm(0640) create_dirs(yes) dir_owner("root") dir_group("adm"));
};
destination d_logserver_net {On the server side we receive the messages:
tcp("1.2.3.4" port(514)
tls(ca_dir("/opt/syslog-ng/etc/syslog-ng/ca.d")
key_file("/opt/syslog-ng/etc/syslog-ng/key.d/client.key")
cert_file("/opt/syslog-ng/etc/syslog-ng/cert.d/client_cert.pem")));
};
log {
source(s_apache_access);
destination(d_logserver_net);
};
source s_apache_net {We are ready. It was not very difficult, was it?
tcp(ip(0.0.0.0) port(1999)
tls( key_file("/opt/syslog-ng/etc/syslog-ng/key.d/syslog-ng.key")
cert_file("/opt/syslog-ng/etc/syslog-ng/cert.d/syslog-ng.cert")
ca_dir("/opt/syslog-ng/etc/syslog-ng/ca.d")) );
};
log {
source(s_apache);
destination(d_apache_access);
};
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 02. 23. 13:34
There are two security related issues about cloud computing and virtualization I am interested in.
1. The are not only spare-time services on the web but many-many business services running in clouds, as well. When you use some SaaS (Software as a Service) for example google aps, linkedin, twitter, facebook or salesforce... you use the http protocol as a remote GUI protocol and not as a conventional content channel. Enterprises should log and secure these channels just like other inhouse applications , but it is not easy because of the encrypted HTTPS. (This train of thought is related with SCB which can be fitting for this problem in the future with an http and an ssl proxy). So, this is a problem. :-)
There are a firewall related effect of SaaS, as well. Namely, companies want to enable and disable these web services. Not URLs, not protocols but services. And the gateway should identify these services with a combination of URL filtering, proxiing and pattern matching. This will be a new approach of firewalls.
2. Security in the cloud. Many companies started to build their own cloud with a high performance hardware or with an even more powerful cluster to run several virtual machines on it. Theoretically, these virtual machines act as real hardware without direct connection between each other. But, as we know very well, every single software have their bugs... so, we shouldn't believe that there is not security gaps in a cloud. But, what is the solution????? There is no concept about this problem. Maybe, we should imagine virtual gateways between virtual ports of virtual machines. This problem become a real big issue when we share a rented cloud with other organisations or when we use a Saas.
Make a comment with your opinion!
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager2010. 02. 17. 13:56
We started the BalaBit Campus Program six years ago with three legs. We established a scholarship, we started a trainee program and we offered our experts for universities as guest speakers.
All of them runs with success, but now I would like to announce something great as for the university education. Last year we started a lab at University of Óbuda with five other Hungarian security companies (Kuert, VirusBuster, CERT.hu, e-Group, Security.hu). The subject contained general, product independent security disciples presented by this six Hungarian companies.
AND!!! Considering the last year interest this year the subject has been started as a recitation without limit of number. We are very proud of it!!!
AND!!! Considering the last year interest this year the subject has been started as a recitation without limit of number. We are very proud of it!!!
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager