Related pages
Bloggers
Tags
News
2009. 12. 08. 11:32:00.
I have just sent out the announcement of SSB 1.1. This is the first feature release that we have released so far. (Feature releases have shorter development and also support period, while they introduce new features earlier for interested parties. See more details on our new release structure.)
For detailed description on new features and changes see the "What is new in SSB 1.1" document. For the inpatients here is the highlight of the most important changes:
High Availability improvements
SSB is able to use the production interfaces (external/management) as redundant heartbeat links between the nodes of the SSB cluster. This will prevent split-brain scenarios in case the primary HA link fails.
SSB is able to monitor the next-hop routers from the nodes and trigger takeovers if the monitored routers become unaccessible from the master node, while available from slave node.
Enhanced reporting and statistics
It is possible to display the number of collected log messages as bar or pie charts in the reports and on the dashboards. Reports and statistics include charts on Top Talkers, Top Host names, Top Programs and others.
Users are also able to create and customize periodic reports to include charts and lists from syslog-ng statistics, as well as statistics on the collected log messages (including messages stored in SQL databases and the indexed logspaces).
Searching and indexing
Users have now the ability to create and save reusable filters on the Search pages. Permissions can be assigned to global filters to grant access to certain log messages for users who have no SSB access otherwise.
Users can now display the distribution of the log messages based on certain parameters (like sender address, hostname, program, facility etc.) as graphical charts. The charts can be included in custom periodic reports as well to provide more details on log messages.
Search queries on logspaces could be restricted to search only certain columns, but not in all part of the message.
It is possible to index and search encrypted logspaces. Decryption keys can be assigned to logspaces or users can upload keys to their personal (passphrase protected) key-stores.
Message classification extensions
SSB 1.1 supports version 3.1 of the pattern database format and functionality, including full tagging and value assignment support. The tags and name/value pairs assigned to log messages can be used in logpath filters, and also stored and searched later.
Besides introducing new features we have somewhat changed our internal development processes. We started using new development methodology: scrum and tried to use more automatic test systems in our Q&A process besides manual testing. We hope that these changes will lead to improved quality and to releases much more on time.
As always feedbacks and suggestions are always welcome.
Enjoy! :)
For detailed description on new features and changes see the "What is new in SSB 1.1" document. For the inpatients here is the highlight of the most important changes:
High Availability improvements
SSB is able to use the production interfaces (external/management) as redundant heartbeat links between the nodes of the SSB cluster. This will prevent split-brain scenarios in case the primary HA link fails.
SSB is able to monitor the next-hop routers from the nodes and trigger takeovers if the monitored routers become unaccessible from the master node, while available from slave node.
Enhanced reporting and statistics
It is possible to display the number of collected log messages as bar or pie charts in the reports and on the dashboards. Reports and statistics include charts on Top Talkers, Top Host names, Top Programs and others.
Users are also able to create and customize periodic reports to include charts and lists from syslog-ng statistics, as well as statistics on the collected log messages (including messages stored in SQL databases and the indexed logspaces).
Searching and indexing
Users have now the ability to create and save reusable filters on the Search pages. Permissions can be assigned to global filters to grant access to certain log messages for users who have no SSB access otherwise.
Users can now display the distribution of the log messages based on certain parameters (like sender address, hostname, program, facility etc.) as graphical charts. The charts can be included in custom periodic reports as well to provide more details on log messages.
Search queries on logspaces could be restricted to search only certain columns, but not in all part of the message.
It is possible to index and search encrypted logspaces. Decryption keys can be assigned to logspaces or users can upload keys to their personal (passphrase protected) key-stores.
Message classification extensions
SSB 1.1 supports version 3.1 of the pattern database format and functionality, including full tagging and value assignment support. The tags and name/value pairs assigned to log messages can be used in logpath filters, and also stored and searched later.
Besides introducing new features we have somewhat changed our internal development processes. We started using new development methodology: scrum and tried to use more automatic test systems in our Q&A process besides manual testing. We hope that these changes will lead to improved quality and to releases much more on time.
As always feedbacks and suggestions are always welcome.
Enjoy! :)
Posted by 
Márton Illés, Product Architect
Márton Illés, Product Architect2010. 03. 15. 16:07
I am happy to announce that we have a new colleague in our German Team in Munich. His name is Frank Mild and has more than 18 years of sales and business development experience. Frank helps Dietmar, our other Sales Manager, to support more and more sales activities in the German speaking countries.
Frank and Dietmar have already been working together as colleagues, therefore Frank was not only a good recomendation for us, but I am also sure that they will be a good team for further successes.
Our aim is to have even stronger sales support in DACH and to be able to reach even more potential business partners and customers, to whom our products can be useful.
Frank and Dietmar have already been working together as colleagues, therefore Frank was not only a good recomendation for us, but I am also sure that they will be a good team for further successes.
Our aim is to have even stronger sales support in DACH and to be able to reach even more potential business partners and customers, to whom our products can be useful.
Posted by Enikő Visky, Regional Director
Enikő Visky, Regional Director2010. 03. 10. 10:09
We have moved forward again. From 2010 we issue CPE certificates about our trainings. if you need that, just indicate on the traning entry-form. The certificate do not contain CPE points, but hours, that helps everybody counting the points. It can be different for every profession. In addition CPE points are maximized if the training is product specific. The certified hours are 6 netto hours a day, therefore we issue 6 hours for SSB, 12 hours for syslog-ng PE and 6 hours for SCB. Fur further info wisit ISACA website.
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 03. 08. 11:13
It was in November 2007 when the initial commit of X Control Box was pushed to our VC server -- the idea was to create a framework based on the current Shell Control Box codebase that can act as a base for all our future appliance-like products. We took the chance to rewrite everything that needed to be rewritten: we added proper widgeting for our interface elements, cleared up the CSS code and the browser-server communication, the search interfaces, the whole permission system and added detailed changelogging for everything that can be changed on the UI. The resulting codebased proved to be usable: it took only a couple of weeks to create the initial pre-alpha version of syslog-ng Store Box, and ever since then we've really benefited from having a well-usable common framework into which 2630 commits have added 300k+ lines of code and templates by now.
The video below shows this progress from the beginning to the current state using the amazing visualization tool Gource. Make sure you click over to YouTube and watch it in high quality, it really worth it!
The video below shows this progress from the beginning to the current state using the amazing visualization tool Gource. Make sure you click over to YouTube and watch it in high quality, it really worth it!
Posted by Péter Gyöngyösi, Senior Developer
Péter Gyöngyösi, Senior Developer2010. 03. 06. 15:45
Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called "afsocket".
The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory.
By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL.
And the same plugin framework will enable me to create a wide variety of plugins. My ideas:
Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :)
The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory.
By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL.
And the same plugin framework will enable me to create a wide variety of plugins. My ideas:
- Plugins for all syslog-ng components (source, destination, filter, rewrite, parser)
- Python scriptability (a simple correllation engine in Python?)
- macro transformation functions, for example: $(stripslashes $macro), usable anywhere in templates and stripslashes a plugin that is invoked whenever such an expansion occurs
- Hooks for transforming the log message as it enters syslog-ng (to fix parsing errors for example),
Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :)
Posted by Balázs Scheidler, Chief Executive Officer
Balázs Scheidler, Chief Executive Officer2010. 03. 06. 07:32
Things have been a little rough last couple of months, that's why I haven't posted here. I'm in a rush right now as well, but I just wanted to let you know that I have started working on modularizing syslog-ng.
It is only a preliminary prototype, and as of now it doesn't compile, but the way it's going to work is already visible: each plugin will have its own plugin and with some trickery the large syslog-ng.conf parser will call out to the plugin parser. The user will recognize such a plugin as an integral part of syslog-ng.
E.g. this is a sample configuration file:
@version: 3.0
@module: dummy
...
destination d_dummy { dummy(dummy_opt(yes)); };
...
See the dummy plugin code in my git repository, in the "plugins" branch. Please note that that branch is going to be rebased a couple of times yet, I've released it in the spirit of "release early, release often".
I hope to get some of the recent contributions into plugins, instead of bloating the core syslog-ng code. For example output colorization. I'm also thinking about adding built-in scripting support via Python.
It is only a preliminary prototype, and as of now it doesn't compile, but the way it's going to work is already visible: each plugin will have its own plugin and with some trickery the large syslog-ng.conf parser will call out to the plugin parser. The user will recognize such a plugin as an integral part of syslog-ng.
E.g. this is a sample configuration file:
@version: 3.0
@module: dummy
...
destination d_dummy { dummy(dummy_opt(yes)); };
...
See the dummy plugin code in my git repository, in the "plugins" branch. Please note that that branch is going to be rebased a couple of times yet, I've released it in the spirit of "release early, release often".
I hope to get some of the recent contributions into plugins, instead of bloating the core syslog-ng code. For example output colorization. I'm also thinking about adding built-in scripting support via Python.
Posted by Balázs Scheidler, Chief Executive Officer
Balázs Scheidler, Chief Executive Officer2010. 03. 05. 14:26
I have just found a bug in syslog-ng OSE 3.1beta2. When you use keep_hostname(yes) in the options, then at those sources where this field is empty (e.g. messages arriving on /dev/log), it eliminates the hostname, somehow this way:
After the patch:
Not a big bug, but was very embrassing. The patch is in Bazsi's git tree out there. Thank you Bazsi!
I wish you a pleasant weekend!
myuser@myhost:~$ logger hello
myuser@myhost:~$ grep hello /var/log/syslog
myuser@myhost:~$ 2010-03-05T14:05:33+01:00 myuser[]: hello
After the patch:
myuser@myhost:~$ logger hello
myuser@myhost:~$ grep hello /var/log/syslog
myuser@myhost:~$ 2010-03-05T14:05:33+01:00 myhost myuser[]: hello
Not a big bug, but was very embrassing. The patch is in Bazsi's git tree out there. Thank you Bazsi!
I wish you a pleasant weekend!
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 03. 05. 13:58
It is well known that people and companies can get very innovative if they wish to cut the costs, and it is especially great if this results in a smart solution.
Making a market survey or buying an existing one is not cheap, and getting one for free can be a large benefit for smaller companies.
I felt handicapped from the start that as a small Hungarian company, we haven't had much information about the global market we were aiming at. Purchasing a global market study cost much more than what we could have afforded.
I was hoping that all this will change when we become a bigger company. And now, although we are still not that big, we could afford to buy a study. But there is always something that seems to be more important and promises imminent benefits. I have realized that we will not spend on that sort of thing until its price is higher than the margin of error in our marketing budgets.
But if everything goes well, this will not even be necessary, because our latest initiative offers free market data for everyone.
Communities have always had high priorities for BalaBit: they are part of our philosophy, our strategy and everyday life. We have always been an active member of the open source software community, and our products are based open formats and open standards.
The Open IT Survey 2010 project offers nothing less than a market survey to be filled by community members who are free to use the results. Participants can complete the anonymous questionnaire in a few minutes, and gain access to global marketing data about IT market trends.
The members of the community have the needed knowledge, we should just collect and share it for everyone's benefit. The open-source movement owes its success to the community, and we are positive that communities are important in shaping the future.
The Open IT Survey wishes to collect the last year's results and this year's expectations, focusing on the performance and trends of the IT and IT security fields. The project has been launched only recently; the first results will be available at the beginning of April at the openITsurvey2010.balabit.com site. According to the organizers, the target group is "everyone who is interested in the results."
The project builds its communication heavily on the community; participants can recommend the project to others on the Facebook Community Portal, as well.
We would like the project to become a regular survey, and gradually collect and share more and more information to everyone, therefore we count on the community's feedback and opinion. Comments and ideas are welcome on the Facebook page.
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager2010. 03. 02. 17:10
I start a new series on Hungarian sights to inform you about our country. The first post is about the Cave Bath which is the only thermal cave in Europe. Some weeks ago I was there with my girlfriend and it was really impressive.
The Cave Bath is a thermal bath in a natural cave in Miskolctapolca, which is part of the city of Miskolc, Hungary, and is unique in Europe.
The thermal water (temperature: 30°C, 86°F) is reputed to cure pain in the joints, and since it has lower salt content than most thermal waters (around 1000 mg/litre), people can bathe in it for much longer, practically for an unlimited amount of time. The Cave Bath can be visited all year long except for January.
The cave and the thermal spring have been known since ancient times, but Tapolca became a popular bathing place only after the Ottoman occupation of Hungary (16th-17th century). During this time the area belonged to the Greek Orthodox abbey of Görömböly; the development of Tapolca into a bathing place was the idea of the abbot in 1711. He also brought doctors from Kassa, today Koąice, Slovakia to examine the beneficial effects of the water. Three pools and an inn were constructed in 1723. The cave itself was not used yet, as the pools were outside. The water was colder than it is now, because the cold water springs of Tapolca (now playing an important role in providing Miskolc with drinkable water) were used too. By the mid-18th century, after a short period of popularity, the bath was neglected and by the 1800s the buildings were in ruins.
In 1837 the new abbot of Görömböly had the buildings restored and expanded. He also had the first indoor pool (though still outside the cave) built, but only for wealthy guests.
In the early 20th century the growing city Miskolc bought the area from the Greek Orthodox Church not only because of the thermal water but also because of the drinking water source (which now provides half of the city's water supply).
In the following years, new public baths were built. In 1934 Tapolca was officially recognised as a spa town. In 1939 the construction of a new bathing house began. During the construction several archaeological findings were unearthed, and a new, previously unknown water spring was found with a water temperature of 31.5°C. The thermal bath was opened in 1941 but the Cave Bath itself was opened only on 14 May 1959.
Since then the bath complex has been expanded several times. The outdoor pool and the characteristic, seashell-shaped roof before it were built in 1969. In the 1980s new rooms and corridors were built, and warmer pools (34°C and 36°C) were constructed. The newest expansion of the bath complex started in 1998.
Posted by Attila Kiss, Marketing Manager
Attila Kiss, Marketing Manager2010. 02. 26. 14:21
I have finished the first online training. We used webex and phone conference. I must tell it was very tiring, but fun. Hopefully there will be more and more training whit this green way of teaching.
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant2010. 02. 24. 10:26
Let's imagine a webserver with many virtual host. It is jailed to a chroot, but it could be on a virtual machine (even a real one). I do want to collect the logs with the following requirements:
- No messages available by apache
- No need to handle hundres of sources. Transfer every log on one channel (okay let's use two;-)
- No need to rotate and support archive
- Use secure protocoll, but I guess I do not have to even mention;-)
mknod /var/log/apache/access.log pNow set all the virtualhost to use them:
mknod /var/log/apache/error.log p
ErrorLog /var/log/apache2/error.logImportant, that all the virtual hosts will use these nodes, therefor logs are not stored in the apache chroot any more. Now we have to read messages some way. The only small problem is apache uses Common Log Format, which is fare from any standard syslog format. Fortunateley it is possible to modify it in apache conf. The original looks this:
LogLevel debug
CustomLog /var/log/apache2/access.log combined
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedLet's change it to comply to syslog RFC mine looks this:
LogFormat "<123>Jan 12 12:12:12 %v apache[666]: %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combinedDo not care about strange PRI field and the fixed timestamp neither the funny PID. I only used it because google indexing;-) Which really count is hostname, which continas the name of the virtual host (%v = virutal host) and the original combined message is delivered at the MESSAGE field ($MSGONLY). So now we are ready with the apache side, let's focus on the syslog-ng side. The first step is reading the two pipes:
source s_apache_access {On the destination (writin side) we simple sore in different files by hostname field:
pipe("/var/log/apache2/access.log);
};
destination d_apache_access {I like solving archiving under the same time, therefore the second destination does it. Now I have to wire the client and the server side:
file("/var/log/apache2/$FULLHOST" template("$MSGONLYn") template-escape(no) owner("root") group("adm") perm(0640));
file("/var/log/archive/$R_YEAR/apache/$R_MONTH/$FULLHOST.$R_DAY" template("$MSGONLYn") template-escape(no) owner("root") group("adm") perm(0640) create_dirs(yes) dir_owner("root") dir_group("adm"));
};
destination d_logserver_net {On the server side we receive the messages:
tcp("1.2.3.4" port(514)
tls(ca_dir("/opt/syslog-ng/etc/syslog-ng/ca.d")
key_file("/opt/syslog-ng/etc/syslog-ng/key.d/client.key")
cert_file("/opt/syslog-ng/etc/syslog-ng/cert.d/client_cert.pem")));
};
log {
source(s_apache_access);
destination(d_logserver_net);
};
source s_apache_net {We are ready. It was not very difficult, was it?
tcp(ip(0.0.0.0) port(1999)
tls( key_file("/opt/syslog-ng/etc/syslog-ng/key.d/syslog-ng.key")
cert_file("/opt/syslog-ng/etc/syslog-ng/cert.d/syslog-ng.cert")
ca_dir("/opt/syslog-ng/etc/syslog-ng/ca.d")) );
};
log {
source(s_apache);
destination(d_apache_access);
};
Posted by Péter Höltzl, IT Security Consultant
Péter Höltzl, IT Security Consultant