Profiel van RobbyRobby van DammeFoto'sWeblogLijsten Extra Help

Weblog


    29 augustus

    TODOList

    For quite a while now, I've been using TodoList to manage my personal list of actions to be done.
    It's a freeware tool that provides quite a few handy features to tune it to your personal needs and I'm quite happy with it.

    A new version was recently released.  You can find it on the site of the CodeProject (sourcecode included): http://www.codeproject.com/tools/todolist2.asp

    24 augustus

    Popular Excel Formulas

    Microsoft published a list of commonly used formulas on their site.  Handy if you don't want to search the documentation!

    http://office.microsoft.com/en-us/assistance/HP052001271033.aspx

    21 augustus

    Paging in a Swing JTable

    After quite a few years in Microsoft and .Net development, I'm currently engaged in a J2EE project.  Wow, that's something new for a change. 

    Although I've always tried to follow developments 'on the other side' from the sideline, I must admit that jumping into it (or better: being thrown in) can actually give you a refreshing view on how things or done.  To avoid any irrelevant comments,  I'm going to leave the J2EE vs .Net (philosophical) discussion for another time, and instead focus on a little issue I had to solve recently.

    The situation:

    • We have Swing application that displays information in a tabular format using (a subclass of) JTables (a bit dull maybe,  but then I never promised this was going to be a world-breaking technological breaktrough, right ;-)
    • The JTables are fed with a subclass of TableModel that does some extra tricks like sorting, cellformatting, etc
    • Everythings seems to work fine until we started testing with 'relevant' amounts of data (read: 1,000,000 + records).  Results: it takes a few minutes to get the output on the screen + memory consumption is huge.

    Being not so lucky with our applications responsetimes, we decided something had to be done.

    Why was this list so slow?

    • All data was loaded in memory (this explains the extreme memory consumption)
    • Objects had to be created for all data (we use Hibernate)
    • All data had to be streamed from the db server to my development machine (not great for speed either)

    Of course only an idiot user would try to get all sales orders (just an example) from the database, and good application design can make sure that your users make efficient queries in most cases, but that doesn't mean that any different behaviour should be punished by responsetimes that dry out the coffee machine in no time.

    The thinking process:

    As far as I remember the solution for this problem lies in the concept of paging: just retrieve the data that the user wants to see and nothing more.  This also means you have to sometimes get extra data from the db server (or appserver) if your user scrolls down the list.

    Big was my surprise that there wasn't really an out-of-the-box solution (not even a copy-paste solution) for this problem.  Anyone that knows one, please don't hesitate to extend my (rather limited) knowledge of the J2EE platform.

    So we dug in, and tried to build a solution ourselves.
    What we eventually came up with was an adapted TableModel class to takes care of the paging.

    This is how it works:

    • Our TableModel is given an object that can retrieve data from the data source (through a well-defined interface), instead of the data itself.
      Next to that, a definition of query that should be executed is also given.  This definition contains what data should be retrieved (e.g. SalesOrders), how the data should be filtered (e.g. for a certain customer) and how it should be sorted.
    • Next to the actual data, the TableModel also contains a private List of Id's (PK in the datasource) for the objects that should be part of the tablemodel. 
      This is a far more limited set of data than the complete set of columns for all these objects.
      This list of Id's is re-initialized whenever the filtering or sorting definition for the model's data changes.
      The sequence of the elements in the list indicates the sequence of the data.
      So we ended up with a list of strings (our Id's are all strings):
    • private List<String> idList;
    • The actual data is only retrieved in pages of a specific, limited size (100 records in our case).  Pages are retrieved when required by the user (e.g. when the Jtable is displayed, when the user scroll down, etc). In our code this data is cache in a variable called 'rowCache'
    • protected HashMap rowCache;
    • As the JTable interacts with the TableModel via the getValueAt method to retrieve its content, this is method where you have to get data from the rowCache and also store extra pages in the cache.
      Our implementation looks like this (simplified):

      public Object getValueAt(int rowIndex, int columnIndex) { if(!rowCache.containsKey(rowIndex)) { int firstRow = rowIndex - (rowIndex % pageSize); // determine last row to get: either a complete page, or until the end of the set int lastRow = firstRow + pageSize; lastRow = idList.size() < lastRow? idList.size(): lastRow; AddPageToRowCache(firstRow, lastRow); } // retrieve the value from the cached objects beanWrapper.setWrappedInstance(rowCache.get(rowIndex)); return beanWrapper.getPropertyValue(getColumnPropertyNames()[columnIndex]); } else { return null; } }
      The rowCache uses the row index as a key, and it"s therefore very easy to verify if the requested row is already retrieved.
      If this is not the case, the boundaries of the page to retrieve are calculated and the method AddPageToRowCache is called to retrieve the new page.
      Once the new page is retrieved, an object is bit of hocus-pocus is performed to retrieve the data from the obect at the correct location (I'll leave the explanation of this code for another time)
    • The AddPageToRowCache() method takes care of the retrieval of new pages in the cache:
    • private void AddPageToRowCache(int firstRow, int lastRow) { // get the objects that correspond to the ids in the page GetObjects(idList.subList(firstRow, lastRow)); // put the objects in the cache and identify each object by its rownumber int rowNum = firstRow; for (Iterator iter = tempCol.iterator(); iter.hasNext();) { rowCache.put(rowNum, iter.next()); rowNum++; } }
    • GetRowCount is even simpler: just get the amount of IDs
    • public int getRowCount() { return idList.size(); }

    Conclusions

    A big advantage of working with list of IDs is that you will retrieve a consistent set of data for your user, even if data has changed in the backend store (watch out for deleted data though)
    Our screens now have an acceptable response time, as they load the first page pretty fast.
    Loading extra pages from the datasource isn't as fast as loading from memory, but that is a small cost to pay.
    Besides, good functional design of the application makes sure that the users get the correct data in the first page in 95% of the cases.

    Syntax highlighting in Windows Live Writer

    Cristoph De Baene, an ex-colleague of mine, created a plug-in for Windows Live Write that allow easy insertion of code snippets in the texts you create using Windows Live Writer.

    I tested it with some Java code and that seems to work fine. 
    The tool is definitely usable and seems quite stable.  As Christoph indicates himself there may be a few nice features to add in the future, but that doesn't make the current version worth trying.

    This is the link: http://www.istaysharp.net/index.php?title=SyntaxCo...

    19 augustus

    RSS Readers

    I've been using RssBandit for the last 2 years or so, and I always found it quite an easy to use RSS reader.  Lately, I did notice that it was often unable to connect to my own blog on windows live spaces.  This seems to be a chronic issue and since the update frequency of  RssBandit is rather low, I decided to see if I could find another solution.

    First option: Google Reader (http://www.google.be/reader).

    • First impression : not bad, cool web technology.
    • Second impression:  not for me.  I do like to read my feeds when I'm offline, and the fact that you can't mark all feeds read in one go is quite annoying.

    So back to Google to search for a desktop aggregator that works like RSS Bandit, but doesn't have its problems.
    Currently the best I found is SharpReader (http://www.sharpreader.net).

    • They do have regular updates (last one dates from August 2nd)
    • No problems with Windows Live feeds

    I'll keep you posted if I change my mind, but I think I'll give this one a try.

    By the way, the migration of my categories and feed subscriptions in RSS Bandit to SharpReader (via an opml file) went flawlessly: thanks to the authors, because I wouldn't have liked typing in all those feed URLs!

    Windows Live Writer

    This is my first blog item written with Windows Live Writer.   Windows Live Writer is a desktop editing tool that allows easy creation/publishing of blog posts on your blog.  It features WYSIWIG editing and photo publishing.

    I haven't tested it with any other blog software, but as you can read this post on the web it must have worked on live spaces!

    You can download the beta version from the team's blog: http://windowslivewriter.spaces.live.com/ 

    Addendum:
    After my succesfull post, I went on to more advanced topics: add a photo to my blog post.  I added to photo from my PC, and tried publish: Result : nada.  I got a Network Connection Error.

    I found some explanation that the weblog has to support the newMediaObject API.  Apparently that's not the case.
    And you know what's really weird: the photo did get onto my site (as a new photo album).  Apparently the guys of the Live Writer team has yet some work to do.

    I wish them good luck, as I quite like this idea of a simple, yet flexible tool for web publications.

     
    *