Published on
15. February 2010 by
Michael Friedrich in
Core, Development, IDOUtils, Misc and Technical.
Tags: 1.0.1, Database, Development, git, Icinga, IDOUtils, ocilib, oracle.
It’s been a while since I made several changes to the initial Oracle implementation in Icinga IDOUtils. Code has been split, first start of using prepared statements and binded params with ocilib and some other changes to the code.
In the last few weeks I have been investigating a lot on how to implement more improvements and optimize the critical path of data input from Icinga Core.
I want to start with IDOUtils Oracle, more information on other improvements for Icinga and IDOUtils will follow :)
Oracle implementation splits up into several parts taken care of:
- Rewrite all queries to prepared statements and bind params at runtime
- Add dynamic procedures for DELETE statements
- Drop autoincrement emulation by one sequence and insert triggers
- Add sequences for each table and use INSERT INTO … (id, …) VALUES (seq_name.nextval, …)
- Add RETURNING id INTO :id for INSERT statements to save one round trip
- MERGE does not support returning INTO, added SELECT seq_name.currval query instead for fetching last inserted id
- Rewrite selecting cached objects from DB
The rewritten queries are divided as follows:
- 1x SELECT latest data time as is (called only at startup)
- dynamic procedure for DELETE on table by instance_id called at startup for cleaning config/status
- dynamic procedure for DELETE on tably by instance_id, field compared to time called during periodic cleanup
- all other queries are prepared with their own statement handler
- 4x DELETE
- 52x MERGE
- 9x INSERT
- 9x UPDATE
- 5x SELECT
This summarizes into about 8000 lines (+) and 2000 lines (-) of code modifications :-)
Furthermore I have been thinking on how to provide an upgrade path for all existing IDOUtils Oracle users. Importing data using the newly applied sequences might lead into errors regarding currval of each sequence. A basic upgrade procedure has been provided already – if you want to try, get the latest GIT master.
Stay tuned for more interesting stories to tell :)
… and watch out for Icinga 1.0.1 and fresh IDOUtils Oracle!
We’re moving our demo systems to a new place. This needs a couple of hours, sorry for the downtime.
Affected systems:
The systems are up and running again. Happy testing!
Kind Regards,
Marius.
Published on
29. January 2010 by
Michael Friedrich in
API, Core, Development, Documentation, Technical and Webinterface.
Tags: API, Database, Development, Docbook, Documentation, Fork, git, Icinga, IDOUtils, oracle, PostgreSQL, release, team, tracker, web.
Currently the Icinga documentation is only available in English & German, we need to expand this to include many other languages. This is where you (the community) can help out! if you are already a translator for another project or would like to get involved in becoming one, then the Icinga Team welcomes you to the challenge…
The Icinga documentation is currently in XML format, a good application for editing the XML files is called XMLmind the personal edition is free to download. Next get yourself a fresh copy of the current documentation from the Icinga git repositories…
git clone git://git.icinga.org/icinga-doc.git
Once you have downloaded this, navigate your way to either the de or en directory and there you will find all the documentation XML files awaiting you!
So if this sounds like a challenge you would like to participate in, then please contact us at info@icinga.org to get your translations included into future releases of Icinga…
Well 2009 has certainly been a very exciting year for the Icinga project, from what started as an idea has become a reality! But this would not have been without your support. What has been achieved in such a short period is due to feedback from you, the user! This is what has shaped the project and made it what it is…
Behind the scenes there is a very dedicated team that has strived to achieve the impossible, and has delivered! with improvements in supporting PostgreSQL & Oracle. Also with current development to a new web UI that will both improve the look and functionality. A lot of work has been done with documentation, currently there is full docs for German & English with work being done on translation to Spanish.
So what will 2010 bring to the Icinga project?
From all of us of the Icinga Team, we wish you a safe and happy new year…
Published on
16. December 2009 by
Michael Friedrich in
Releases.
Tags: 1.0, API, Database, Development, Docbook, Documentation, Download, Icinga, IDOUtils, mysql, oracle, release, Support, web, Web 0.9.1 alpha.
December 16 2009: Today the Icinga Team releases the Icinga Core 1.0. This is a milestone for both the team and the project as a whole. After many months of hard work we are proud to bring you a stable, alternative monitoring solution. This release includes many changes as suggested by the community and in particular the inclusion of Oracle in IDOUtils.
With just as many new improvements, Icinga Web UI has hit release 0.9.1 alpha. We have added a makefile for easier installation and fixed installation permission and cache problems. More changes are still to come, including an ExtJS update to 3.0.3. See below for the full list of new developments across Icinga Core, API, Docs and Web.
As we are always eager to keep the momentum going, we have decided to release the stable Icinga Core alongside the Icinga Web 0.9.1 alpha. These two will converge again in the coming months to a uniform release status. Till then, we hope you like the latest improvements.
Core:
- Improved IDOUtils with Oracle
Added prepared statements for most called queries
Split code into ocilib OR libdbi, to allow oracle to decide which rdbm lib will be used during configuration
- idoutils: fixed duplicate rows in table system commands, timed events, timed event queue (missing unique keys)
- idoutils: added upgrade path/sql queries for unique key failure – check docs for more information
- idoutils: changed default data_processing_options in idomod.cfg
- idoutils: fixed this version and perl path generation in db install scripts
- idoutils: fixed save custom variables segfault
Docs:
- Updates and fixes for quickstart guides
- New section on upgrading Icinga & IDOUtils
- Revised section for Icinga Web
API:
- Restructured DB access for upcoming RDBM support
- Made several fixes for table prefix, exception handling
- Started a ‘how-to’ guide for upcoming documentation
Web:
- Added makefile for easier installation
- Fixed installation permission and cache problems
- Modified .htaccess
- Removed yui
- Removed php notice warnings (isset, undef vars)
- In the process of changing API result keys to uppercase
- In the process of updating ExtJS to 3.0.3
- Introducing commands through the web
Should you find any issues, please report them to the following links:
As always we look forward to your feedback, so feel free to drop us a comment.
I am sure there will always be much discussion around the IDO-/NDOUtils database model. In my opinion there are two major problems with the model at the end of the day:
- Normalization
The tables have a lot of redundant information regarding their unique id’s. Different object types have a corresponding id and different object tables. To query a bunch of data you need to join the object and instance tables in most of the cases. This makes it hard to find a specific value without knowledge of the model.
- Prefix and table names
Icinga or Nagios as a prefix for every table makes no sense. The reason is that every supported database has schemas to store the tables and that is a better place to distinguish this. On the other hand we have a problem with Oracle to store tables with more than 30 characters.
In addition to that, some other problems like blocking, broker finetuning and loss of performance due to a lack of correct indices on the tables is hard work to do. At the moment, there is no time to change the whole model for every supported database and there is also a chance that the community has interesting suggestions worth first considering.
As a first step we want to introduce a new view layer based on the existing ndo model. A view is a “virtual” database object that queries the original data in the defined target table. I know this is not 100% true for every database, because we have various view types in oracle, but for now it is the only important thing. With this first early version we tried to solve these three issues:
- Every object table (which has an own object_type in the object_table) includes the correct object_id and a join to the instance table
- Every table is grouped into a configuration, historical and runtime area, which makes it easier to find a way through the model
- Every table has a grouped tablename for example ic_hosts_escs (icinga, configuration, hosts, escalation)
In an early stage of modeling I recognized that this will not be the final step and I decided to develop a code generator based on Java. Because it is a drop away thing, there was no focus on performance or style, so please forgive me. You can download the generator here as it is without any warranty.
Loading ...
You can download the first version of the view model here and run it against your database just check that your prefix is correct. We are looking forward to your feedback and also a rating on this idea.
Are you a developer for PostgreSQL? Do you have spare time to dedicate to a good cause? Do you enjoy working in a team environment? Is coding your focus?
Well no, this is not a job offer! but a request for a competent developer who is familiar with the ways of PostgreSQL & C/C++ to either join or can assist with future development of PostgreSQL within Icinga. Now is that an offer you can’t refuse?
So if this looks like something that you would like to contribute to, then please contact Michael Friedrich with your interest …
Michael is also available on IRC #icinga-devel
Hi folks,
the day has come: the new web interface is online!
Let’s visit http://alpha.icinga.org/ and log in as guest by using the password guestuser.
Sneak and poke around and give us feedback at icinga-devel@lists.sourceforge.net!
Have fun,
Christian
Loading...
Recent Comments