Author Archives: Howard

Weathermap 0.98a adapted for Cacti 1.x

Just in case you weren’t aware, there is a modified version of Weathermap 0.98a, created by Thomas Urban that works with Cacti 1.0+. For a number of reasons, it will probably stay as a separate version, but if you need Weathermap and modern Cacti (and you should upgrade! Cacti 0.8.8 is quite old now), this is your best bet.

https://forums.cacti.net/viewtopic.php?f=16&t=60353

Sponsorship followup

I realised since I was doing one, I might as well set up the other, so there’s now also a Patreon page for Weathermap. They don’t match donations, but you may already have an account with them. Again, it’s set up for small amounts, although there might be a ‘swag’ tier above the current ones, if I ever sort out stickers again.

Also, thanks to alcatron, my first sponsor!

Sponsorship, and (still) progress

Two items of news:

  1. Several people have asked, so you can now sponsor me (and weathermap development) via Github if you’d like to. For the first year, Github will match any donation too, which is very nice of them. I have specifically limited it to ‘buy a coffee/beer’ size of donations. Which leads to…
  2. There is still weathermap development. I just noticed that the last post here about it was that the Cacti 1.x version was dead. Since then I’ve been doing work as I planned to get Weathermap 1.0 working with Cacti 0.8.8. Mark Brugnoli-vinten of the Cacti dev team has kindly volunteered to get that updated to a “new-style” Cacti 1.x plugin and maintain it. It will be a more traditional UI, so anyone who has been playing with the React-based version in github, that’s basically a dead-end. This work is going slowly (see all previous posts about that), but it is moving forward! (there’s a separate branch on github for it)

Weathermap 0.98a released (NO, not that one)

I’ve been sitting on this to gather feedback from testers, and I haven’t had anything negative for a while, so here is Weathermap 0.98a. The previous version has hit around 32000 downloads – so thanks for your continued interest in this transition period!

THIS IS STILL NOT COMPATIBLE WITH CACTI 1.x

This is basically Weathermap 0.98, updated to remove dependencies on the old mysql_ database functions, so that it’ll work with PHP 7, and also a few other bugfixes that are currently in 1.x dev code, but easy to replicate in the older code.

The other big difference, for some people anyway, is that it’s now MIT licensed. This should help out a few folks with either integration or corporate legal hassles.

End of Cacti 1.x plans

I’ve been meaning to write this post for a while, but figured it was time…

I’m no longer working on an update to Weathermap to support Cacti 1.x

As I’ve said before, I don’t use Cacti or Weathermap myself anymore, so there isn’t really much motivation to work on it. It’s a job now, and I have another one of those!

It’s been 14 years, and while it was fun for the first 10 or so of those, it’s never really grown to be a collaborative project, mainly thousands of hours of my personal time (and some money too). There are a few exceptions for specific features along the way, but 5000 forum posts later, I don’t have a network ops job anymore, so it’s going to have to have some other motivation to keep going.

My immediate plans for Weathermap are to split it up into multiple repos, for at least the core map-drawing stuff, and the Cacti plugin, and possibly the editor. I will probably continue working slowly on the core, although I’m more interested in rewriting that in Javascript at this point. I’m not aware of  many issues with the core drawing and data collection, currently.

One of my design goals has been to separate out the Cacti-specific stuff for some time, to make it easier to integrate with other NMS tools (e.g. LibreNMS or Zabbix). That will also make it easier for someone with a vested interest to take on those parts for their platform, without having to deal with the others. The actual core map-drawing and data managing stuff is actually fine, and faster than ever! So someone else could take over the Cacti part – either starting from the 0.98 (“old”) version again, or carrying on with React. That said, there have been only a few outside code contributions to the project over the years. Hopefully that can change.

For anyone interested in taking on that Cacti support – it’s not as huge as it looks, once you take away the map-drawing! It’s managing entries in a few tables, plus some stuff specific to Cacti’s poller (“boost support”), and the datasource plugins that are only relevant to Cacti. A good starting point would probably be cigamit’s patches from a couple of years ago. Ask, and I will go digging in the change history for them.

I’d like to thank everyone who has supported Weathermap over the years – mostly within the Cacti community, but also outside (thanks UKFAST for hosting support in the last year, for example, and Groundwork Open Source all those years ago for a chance to meet with fellow projects). You were definitely in the “fun” part! 

Finally, I think I have enough confirmation now to make 0.98a an actual release version, too. So I will do that later today. That’s 0.98 with appropriate updates to work on PHP 7.

New Hosting…

Things might be a bit wonky for the next day or so as DNS straightens itself out, but the weathermap website has finally moved from shared hosting to a shiny new VPS, thanks to the kind support of UKFAST

The forums will follow shortly, and after all the dust has settled, everything should be noticeably faster. There are some loose ends to resolve currently though – like the manual pages that normally live here are currently missing. As soon as Dreamhost’s server wakes up, I’ll get those restored. [EDIT: These should all be restored now]

Thanks again to Neil and UKFAST for sponsoring the new hosting!

(yes, I am working on the next release too)

13 Years Old!

13 years ago today, the first version of what became PHP Network Weathermap was released… it was a series of versions in Perl, adding functionality to the GRNET Weathermap. After a few of those, and getting tired of explaining how to build CPAN modules, I started re-writing the whole thing in PHP, initially before the Cacti Plugin Architecture existed, as a standalone script sometime in April 2005. Sadly, it looks like the Cacti Forums have had some database corruption over the years, so posts that old seem to be missing.

Not dead yet

I know this is taking a long time, but here’s what I’m up to with Weathermap now:

  • Cacti 1.0 uses a sort-of old-fashioned “download chunks of HTML” approach to its new UI. That meant rewriting the Weathermap Cacti plugin to work in this (undocumented) style, too.
  • I also have wanted to rewrite it anyway for some time into a more object-oriented format which separates the actual actions on the database from the display code. This makes it possible to write automated tests for the action code, and for someone to potentially reuse it a lot more easily in another NMS integration.
  • I was halfway through doing all that, when I realised that if I was going to have to rewrite it anyway, I’d rather do it in a manner than benefits everyone, including non-Cacti users. Particularly as I am personally not really much of a Cacti user nowadays, but also I’ve wanted to make it easier for folks like the LibreNMS project to integrate weathermap for some time now.
  • So: new plan. The user-interface parts of the Cacti plugin will be entirely javascript client-side code. The remaining plugin parts will just form an API that the client-side calls. This means I can ignore the Cacti AJAXy redesign stuff, largely. I’ll also be trying to ensure that the client-side doesn’t need Cacti-specific data. At that point, someone from $OTHER_NMS team can embed the client javascript as-is, and implement a simple JSON API, and they too have integrated Weathermap, including map management. To prove this works, I’m planning to make a standalone web app too, providing a web-based, database-backed manager for people using non-integrated NMSes.
  • The ‘user’ plugin is about 70% done (map cycling remains). The ‘management’ plugin is started but early days. This work is also a lot more fun than trying to untangle someone else’s intent though, so it’s progressing well. The client side of things will be written with Inferno and Redux.

Once again, sorry for the long delay, but I wanted this to be right. I also wanted it to be fun to do – this is my hobby project, without much external contribution[*], so having it feel like a job I hate doing for the last year has been no fun at all. The end result should be worth it.

Also, this is the path the data-picker, and then editor will take in a future release, and quite possibly weathermap itself after that – I already did a little bit of experimentation with that.

* A huge thank you to those contributors though! It is still my goal to build a development group around the project if possible. For me personally, that still means simplifying, modernising, clarifying the internals, to make it easier for others to get started and contribute. I do still see this as the ‘dark ages’ before a brave new world of open source weathermap – it’s just taking longer than I hoped! 🙂

Site and project updates

First of all, apologies for the slow progress, but the “obstacles”* in my life that have been slowing things down are clearing up. The website has a temporary new look, after the previous theme I’ve been using seems to have dramatically changed between updates. This one happens to also match better with the other new change: a forum!

It’s rather empty at the moment. This is supposed to be the worst way to launch a forum, but I can’t see a way around that. I really wanted to have somewhere for people to talk about their weathermap issues that wasn’t necessarily dependent on Cacti. I know lots of Weathermap users are not Cacti user. I also wanted somewhere better than Github Issues to discuss and dissect problems, before they become specific issues on Github. Also, somewhere to try and grow some development community. I’ve been having good discussions about this with some folks recently, and there some great early work by Martin over there about getting going with development stuff.

So progress should speed up a bit for what I’ve decided to call Weathermap 1.0.0 which will have Cacti 1.x (and Cacti 0.8.8) compatibility. Thanks to those who are hitting me with bug reports and even the odd pull request!

And I’m looking forward to the next release after that being hopefully from a group…

* Please don’t tell my new wife I said that!

Cacti 1.0 and Weathermap

Cacti 1.0 was released a few days ago, and with it, a new plugin API. Weathermap (even from github) doesn’t currently work with Cacti 1.0. Unfortunately, at least from a first look, it also seems that it’s not possible to easily have a plugin that supports both Cacti 0.8.x and Cacti 1.x. The Cacti Group’s own plugins have made a clean break, with new versions only supporting 1.0. I think it will be a while before everyone moves to 1.0 (personally I avoid version x.0 of anything, especially with such big internal changes), and that it’s unfair that to get new Weathermap features (or fixes!), you’d need to upgrade Cacti too. So I want to continue supporting 0.8.x with new versions also, at least until Weathermap is also settled down again. With PDO, and PHP 7 support, and the changes related to code improvement, I’d hope that even if I decide to stop supporting Cacti 0.8.x, someone else could continue, after that point.

After a very busy new year, I’m back to thinking about Weathermap again, and will be finishing off the changes for PDO and PHP 7.0 soon. Once that version is released, then I’ll look at Cacti 1.0. It’s likely that Weathermap will be broken into multiple downloads: a ‘core’ with the actual map functionality and probably the CLI, and then a series of extra downloads to provide the editor, Cacti 0.8.x plugin, Cacti 1.x plugin, standalone web app, and LibreNMS plugin. You would download the core plus whichever of the other features you actually need. To get both Cacti versions working, something like this is necessary, and users of other systems don’t need any of that code cluttering up their installation.

There is (as far as I know) one remaining set of failing tests before the next release – all related to colourising icons, and transparency.

5000 downloads of 0.98, and progress on 0.98a

Weathermap 0.98 hit 5000 downloads yesterday… woo!

Also, work is going well on the database changes for 0.98a. The actual database stuff has all been replaced and about 70% tested. Some of the other changes have highlighted some issues though, so I’m working on those now. The solution has been to pull in some of the refactored code from last year, which is tricky in places – it is all interdependent, so you have to try and break bits off cleanly. So now I am working through, weaving the two parts back together again.

Anyway, the end result will be some nicer code that most people won’t care about, and PHP 7 working, which they will! I don’t plan on this taking a lot longer, spare time permitting.

Making Weathermap “really” open source

Although Weathermap has always been available under the GPL, you could reasonably argue that it’s not really open source. The reason it’s GPLed is simply that it was originally written in perl, based off of GRNET’s perl Weathermap, way back at the end of 2004. When I got tired of explaining how CPAN works (to install the perl GD module and dependencies) and switched to PHP, I kept the same license.

Despite being GPLed for 10 years, there have been very few code contributions. I can see why – it’s a fairly big tangled mess of PHP, considering the relatively simple job it does. It was my first PHP application from scratch too, and has lived long enough to be partly PHP 4 and partly PHP 5 style. Even if you do make the effort to read it all, you’ll find it has a few “god objects” which make modification harder than it needs to be. So effectively, it’s a personal project that you could re-use if you really wanted to.

Also the number of users – network engineers, mostly – who can and want to code is pretty small. I don’t want to discourage the few that remain!

Having worked a little on other “real”  open source projects, I really want to change this. For example, when I had an issue with beets, I was able to submit patches, get useful feedback from the project lead, and eventually get my change into the next release. It felt good, was collaborative, and solved my problem permanently too. That’s the kind of experience I want for Weathermap.

The first steps towards this have already happened – moving from subversion to github gets us the infrastructure to do Pull Requests, which is very powerful. There are the beginnings of unit tests, and more on the way as new code is added, to hopefully get to the stage that any PRs can be tested automatically using TravisCI. The code quality is (slowly) improving, and the ‘master’ branch on github has the results of some refactoring already. It may be that only some of that work makes it into a release, but the goal is to get to more and smaller classes, much reduced global variable usage and documented classes!

I have been using CodeClimate to highlight areas to work on, mostly for removing complexity and duplication so far. For example, processing of data source targets in the refactored version is a lot cleaner. Also, as UI code is moved into classes, it makes that easier to test too, which is a big gap currently – the image generation side of things has hundreds of tests. Similarly, there are Vagrant configuration files to automatically produce test environments on 10 or so difference Linux distros.

At the moment, things are still moving around, but once the next couple of releases are complete, I’ll be working on bringing the ‘new’ version and the 0.97-maintenance branches back together, and hopefully have a working version with as much of the new code as possible. Parts (especially the Node class) are still too big and clunky, but there is some useful work already done.

With that in place, I hope we can have a really open source network visualisation tool!

September 2016 – Current issues

tire-fire-1200

Just a quick note about known current issues with Weathermap 0.98:

  1. It doesn’t work with PHP 7.0 – In PHP 7.0, the ancient mysql_xxxxx() functions that WM uses for database access are no longer available. The rest of the world moved on to PDO some time ago, but I didn’t bother because Cacti didn’t. PHP 7.0 is the default in some distros now (Ubuntu 16 at least) though, which makes it a bigger problem. I’m in the process (about 70% through) of fixing this by replacing all the database access code. This has some other benefits for the future, like the possibility of supporting other databases, and better database security (I’m moving to prepared statements everywhere too). I’m also separating out the actual database stuff from the Cacti plugin, so writing a plugin for something else should be easier too. The editor and command-line tool don’t use a database currently, so those are not affected.
  2. (some of) It doesn’t work with PHP 5.3 – Well, this is embarrassing! In fixing a deprecated code issue for PHP 5.6+, I added a problem for PHP 5.3. Luckily it’s a one-line fix, and only affects the command-line ‘weathermap’ tool. If you get PHP Parse errors with 0.98 and PHP 5.3, here is the solution until 0.98a comes out.

Photo Credit: Wisconsin Department of Natural Resources

Links between maps

Introduction

Links between maps, and links to other sites are often requested as ‘new features’, even though they are already possible. Here are the things you need to know:

  • Any node or link can have a URL associated with it, using the INFOURL keyword. Usually, this is a link to a Cacti graph or something similar, but it doesn’t have to be.
  • A node doesn’t have to represent a router or device either.
  • For Cacti in particular, the ‘random’ ID in the weathermap name doesn’t ever change, and is safe to use in (HTML) links.

Simple version

So, putting all that together:

NODE link-to-other-map
    POSITION 300 300
    LABEL Other Map
    INFOURL http://my-cacti-server/plugins/weathermap/weathermap-cacti-plugin.php?action=viewmap&id=4df1356edd3366055c5f

Will make a text link to your other map, using Cacti’s authentication to make sure you are allowed to see it. If you don’t use Cacti, this would just be the URL for the other map as you see it in the browser.

Fancier version

How about having a thumbnail of the other map that you can click on?

As long as both maps are on the same Cacti server, Weathermap already generates a thumbnail to use in the web interface, which you can use too.

Some things you need to think about though: Weathermap updates the maps in the order they are listed in the management page, including the thumbnail updates. So if you want to be sure of getting the latest thumbnail, your master top-level map should be the last one in the list. You can put it in its own group though, so it isn’t all the way down a page of maps, and Weathermap remembers which group you last clicked on, so that shouldn’t be a problem either.

NODE link-to-other-map
    POSITION 300 300
    LABEL Other Map
    ICON 64 64 output/4df1356edd3366055c5f.thumb.png
    INFOURL http://my-cacti-server/plugins/weathermap/weathermap-cacti-plugin.php?action=viewmap&id=4df1356edd3366055c5f

So now we are using the thumbnail image as the icon for our node, and also telling Weathermap to scale it down so that it fits in a 64×64 pixel box.

Final Notes

The only real limitations are that you can’t use a URL for an ICON (this is possible with PHP, but not enabled on all installations. Also, try to avoid using the icon scaling to scale full-size images down to thumbnails, if you are concerned about performance (like updating a set of maps once per minute).

New Site

To get the blog functionality, I’ve switched from a simple static template to WordPress. This currently has limited content – just the text from the previous static page and the manual pages as before. In particular I want to add some images to the the front page – a graphical tool with no pictures isn’t so good! I hope it will all appear in the coming days.

Survey results: this blog!

I recently posted a survey for Weathermap users, asking how they use it, which features they use, and which features they would value most for future versions. One thing that was surprising was requests for features that already exist! So that implies that some better documentation, or tutorial information would also be good.

With that in mind, I’ve converted the static network-weathermap.com site to a blog, so that I can add tutorials here. I’ll try to think of ways to improve the reference documentation too.

I’ll also share some more of the results, once I have more, and have had time to process them.