Posts

Showing posts from 2017

Selective persistence of Oracle Diagnostic Logging (ODL) output

Image
Background and GoalIn any application, logging is widely used for diagnostics and debugging. 
Logging at various "checkpoints" (such as entering with request, exiting with response, error handler) in the application can provide a fairly reliable way to trace the execution path of the application - which a subsequent sweep or count can be used to report on. When the logs are regularly analysed and reported on, anomalies can get flagged up proactively and investigated further. Some examples

Geographical clusters with the biggest concentration of web services

Image
From a data set of approximately 145 million IP addresses running at least one publicly accessible web service (such as a website), I was able to determine these 20 geographic "clusters".


Raw results - countries list with total IP (IPv4) addresses

Background: 
http://weblog.singhpora.com/2017/10/how-many-programmers-does-it-take-to.html


Presented below is a list of countries (country codes) and the total count of live IPv4 addresses where a public facing service (such as a website) might be hosted as counted from the scan data of 1st October 2017

The reason these don't quite add up to anywhere in the ballpark of 4 billion (the total IPv4 address space) is because the data set I used might only be scanning for hosts that run some public service exposed over a TCP port (e.g. a website running on port 80 or 443)
The numbers definitely look incorrect and total up to only 145,430,195 - I will continue to investigate why, but they seem to be in proportion.  It is likely that scans.io are only able to gather data about live IP addresses at the time of the scan as opposed to total allocated ones)

+-------+--------+
|country|ip_count|
+-------+--------+
|     LT|  120718|
|     DZ|  362827|
|     MM|    3494|
|     CI|   18954|

How many programmers does it take to update a Wikipedia page?

Image
......or what it took to count the number of IPv4 addresses in every country (as of 1st October 2017). 
This Sunday, I found that the Wikipedia page on List of countries by IPv4 address allocation was using data from 2012. I wondered what it might take to add more up to date information on that page. During a recent course I attended, I got to know about scans.io - a fascinating project that involves periodically scanning ALL of the IPv4 address space and storing as much of publicly visible metadata about the active addresses as possible (location, ISP, open ports, services running, operating system, vulnerable services running if any). Each daily dump of the IPv4 address space is close to a terabyte. An individual IP address record is represented as a JSon object - part of one of the records is shown here:

Test driven SOA: Tool kit for comprehensive automated test coverage

In this post I am going to share some tools I find useful when developing components for the Oracle service bus - same principles should apply to the integration cloud service as well. 

If we are not test first (or at least test alongside) programming, we are essentially debug later programming (See "Physics of Test Driven Development").

If the enterprise service bus sits in the middle of an organisation's messaging and integration landscape, there are some key architectural principles that help in getting the best out of any service bus solution: It is not the place for business logic but for integration logic i.e. heavy on message transformations and often enrichmentAny operations, message flows or pipelines that the service bus exposes should be stateless and without side effects (ideally). To achieve this, a lot would depend on backend services too - they would ideally need to be idempotent. Exposed interfaces must be designed to be canonical while invoked endpoints abs…

Enterprise World Wide Cloud - Notes from the Oracle #PaaSForum 2017

Image
This must be one of the last posts on #PaaSForum 2017 that anyone has written. Too late but I hope not too little, as it gives us an opportunity to reflect upon the material presented there. I heartily thank Jürgen Kress for organising this event on a grand scale, at a beautiful location (Split, Croatia this year), for his invitation, warm welcome and hospitality. Also thanks to his colleagues from Oracle for their help in organising this event and in sharing their deep knowledge.  Further, we gained a wealth of information from leading Oracle professionals from all over the world - I felt privileged to meet so many of them in person!  We started this event on a positive note (Jürgen literally asked us to take our jackets off and get hands-on with some cutting edge Oracle technology!) and ended it full of enthusiasm watching the sunset over Adriatic sea over drinks and conversation.


Although I learnt a lot in the various presentations, please don't assume every idea or interpretati…