Multiple Image Scrolling

I thought through a couple different ways to allow users to see multiple images for a given node, and I decided that the simplest user interaction was just to have the image change to the next one when clicked on. It wasn't actually as simple to implement as it is to use. 

I created a Rectangle class extending the UIObject class, which the EventManager class sends events to. The rectangle also has a reference to the overall PApplet (Processing Applet), so as to control drawing this invisible rectangle on the screen. On a mouse click event, the EventManager sends the event to whatever UIObjects it knows of, and waits for one of them to handle it before resorting to other options. The Rectangle class handles MOUSE_RELEASED events that fall inside the area of the image. 

First roadblock - either the rectangles were responding to clicks indiscriminately, no matter where they were, or they weren't responding at all. Then when debugging through I realized the EventManager had references to a lot more UIObjects than seemed reasonable... until I realized that for every node on every new frame I was creating a new rectangle instead of updating that node's existing one. That lead to thousands of rectangles in a very short time and caused much of the strange click-response I'd been getting. 

Finding the click area itself was a bit of a task, because there aren't clear variables defining it. The function drawing the image is called using local coordinates, but my rectangle needed to be called using world coordinates. This is where my CIS 460 knowledge came in handy, because otherwise I wouldn't have even known the difference let alone how to handle it. I was able to find variables called realX and realY in world coordinates, which I thought would be the solution to my problem, except for some reason only the bottom left side of the image was responding to clicks. A lot of debugging and detective work lead me to find out that that the realX and realY coordinates are actually that location of the center of the node dot, not the top left corner of the image as I had previously thought. So even though my rectangle was drawn in the right place (tested in color, so I'd know where it was), the code deciding which clicks to recognize was handling the wrong area. 

Once I figured that out, the next step was to try and fix it. The image is drawn just to the right of that node dot, but how far? Some quick math got me a rough estimate, except that value changes depending on the zoom level. What solved it for me was the variable dotWidth, which changed along with the zoom. Turns out the image's top left side is approximately one "dot-width" away from the realX and realY coordinates. Since the dot hits at the center of the image, vertically, I subtracted half the height from the y coordinate and finally ended up with the correct "click area." 

The last step was to handle actually making a click do something other than print out "click!" which is what I'd been using to test it until then. After some quick searching around, I found that the ImageSearcher2 object has a next() function. So I called that. Nothing happened. Then I added a line for loadImageURL() and finally I had changing images on clicks! Except when it got to the end of the available images it stopped. So I added a check to restart from the first when it got to the end, and that is the long but exciting story about how I got my last feature working properly. 

I want to add some sort of way to check whether it's displaying the "Image not available" image, since that's annoying. But other than that, I do believe I'm done! 

Things I learned on this section: 

- PhyloWidget's handling of events and how to interact with it

- The image and node rendering processes, the many variables involved, and how much more hacked together it is than the rest of the code I'd been reading 'till now. 

- I'm a lot faster at debugging now, since a lot of this is trial and error, or rather "trial and figure out where the error is."

- Processing's shape-drawing language. I hadn't really gotten to interact much with it yet, and it was a nice introduction to how to draw shapes, specify coordinates and colors, etc. Of course you can't see my shapes, but they're there doing their job. 

With that, and since I just used all three forms of words sounding like "there" in one phrase, I consider this blog post done. 

UBIO Common Name Search...

...is complete! 

I've spent the past 2 days trying to work around strange bugs and get a working common name search, and I've got it. By yesterday I had the code parsing multiple pages in search of the right information and printing out the resulting common name in the console. Today I changed some of the set and get functions relating to labels so that a user could toggle whether the common or scientific name is displayed, and I now have the names actually displaying as labels on the rendered tree. I even figured out how to display a little message "Done searching for images!" just like there's a "Finished Loading Image!" message. 

Here's a sample image from after the name search/display has been done. It can't find every name, but it gets most of them.

I also spent some time reorganizing the toolbar menus so that they feel a little more intuitive and take fewer clicks to get to what you want. I felt the original structure of the toolbar lead to a lot of overlap and confusion about where a desired option was located. I organized my menus as follows: 

Tree

• Layout (rectangular, circular, diagonal, unrooted)
  
• Node (images and other annotations)
  
• Label (unique, clade, show all, common name search)
• Performance (anti-aliasing, animation)
  
• Zoom

Format  

• Font 
  
• Style (size, scaling, angles)
  
• Structure (sorting, flipping, branch lengths, etc.)

I think now the functionality of the program is actually arranged in a way that makes sense based on what the different options control and how they change the display of the tree. I also spaced them out a bit more, since I found that the original toolbar options were crowded into the left corner. Sample image: 

UBIO Bugs

Current issue diagnosis: 

This form on the UBIO search page uses method="get" as opposed to "post", which the 2 image databases used. 

To connect and output data to the form, I create a URLConnection object, but its default method is post, not get (even though everything I've googled says that the default is get). Can't figure out how to change that setting to "get" because that's what I think is causing the output not to go through. 

Ideas, anyone? 

NBII Image Search

In the midst of preparing for my presentation, I am trying to have as much of a finished product as possible, so I have been working on including NBII.gov in my image search. Here's a detailed description of the search process that I wrote out before writing the code, so that I was sure I knew what I was doing before I attempted getting it to work. 

Here's the html for the form I need to "post" to in my Java code. (Note: I have to post it as an image because otherwise blogspot will actually interpret the code and do what it says - crazy.)

First I see that hitting the "go" button executes search_results.php and that I need to use the POST method to send data to the script. The next thing I need to find is that data that the script expects. One field is called "txtSearch" and the other is called "Submit". The value is txtSearch is not given, but the value of Submit is "Go". So assuming that the value of txtSearch is the search term, let's call it "term", I should be sending the string "txtSearch=term&Submit=Go" to http://life.nbii.gov/search_results.php". 

When I get the response, I search for the term "Total images", since it indicates that I'm about to reach the place in the html where the images are listed. From then on, when I get to a line that contains "xt "Xthumbnail", since that indicates that I've gone past the section with the images I want. 

Posting to a .php site

I'm trying to expand my image search to include the nbii.gov image database, but their site is written using php instead of straight html. I don't know what makes it different, but I am pretty sure I'll need to interact with it differently than I did for my morphbank.org search. 

Here's the relevant line of source code: 

form action="search_results.php" method="post" enctype="application/x-www-form-urlencoded" name="frmSearch"

Anyone know about this and have suggestions for how I go about posting a search term to this form? Thanks! 

Displaying Images (2)

Morphbank image search and display is complete! Next: 

- fix ubio common name search bug

- NBII image search (which will most likely involve cleaning up some of the morphbank search code in favor of code reuse)

- common name display capabilities

- explore display settings for layouts other than rectangular

Downloading and Storing Images

I've integrated the image search into PhyloWidget. The images for each node are stored in the class PhyloNode as a LinkedList. Currently, when loading a tree, a search is conducted as each new node is created. This means that the loading process is quite slow, when really it should be possible to conduct the search after drawing the initial text-only display, or in a separate thread while the file continues to be parsed. Not quite sure how to do that, but I know they're options.

Next step is to display the images within the program as opposed to outside it in tiny little JFrame windows. That'll require some UI manipulation, which I was holding off on until the rest of the code was working. I'm also hoping to search some other databases, because Morphbank has very few images available for some categories of organisms -- at least that's true for the ones in the tree files I've been testing with.

Also, the search I'm conducting returns thumbnail size images. This is useful for quick downloading, but I'd also like to download the full size image so that when a user clicks on the image next to a node, they can see it full-size. I'll need to determine how best to access the full size images from the website and then how/when to download it and store it in the node.  

Finally, one of the more amusing issues is the "Image Not Available" image. The Morphbank search only displays results with images, but unfortunately some of those images look like this: 

Obviously that isn't too useful, but I don't know how I'd filter those out. I don't know if Image objects are comparable, because I could save one of these "Not Available" ones and test whether the returned image is equal to it before displaying it. I'll save that to work on later, though, since it's not crucial to the general functioning of my code. 

Harvesting Images

I am now able to search Morpbank.net for any given taxonomic name and return a LinkedList of all the images associated with that search. I have not yet integrated the code into PhyloWidget but intend to do so later today. 

One of the reasons I have not done this is because many revisions were made to the code during March, which I am not interested in including in my version of the code. Since I download the code using an SVN client, I need to figure out how to download the code as it was before those revisions took place. 

Once I am able to do that, the integration will consist of creating a variable to store these images within each node of the tree, determining when to call the search function with the node's taxonomic name, and displaying the resulting images within the current program. 

Of Note: This website listed a few short, easy examples about reading and writing images in Java, and as a novice at this I found it very useful.

http://www.exampledepot.com/egs/javax.imageio/pkg.html

Schedule Update

What follows is the beginning of a revised set of deadlines. I intend to complete the tasks listed here by Tuesday April 21st:

- Pre-parse newick tree files to replace the given number-encoding with the appropriate scientific names, before sending the tree string to the regular PhyloWidget parser

- Retrieve images from Morphbank as MIME attachments, based on searches by scientific name, and cache the images locally

- Incorporate a thumbnail version of the each image into the PhyloWidget tree display at its corresponding node, according to the diagram posted previously. (If time, allow multiple images per node and create arrows so that the user can scroll through the thumbnails for each.)

After this deadline, I plan to return to the current bug in my common name search to complete that portion of the code. I will also look at the way that PhyloWidget draws the trees with the alternative layouts, to determine how best to incorporate the images in those cases. Finally, I will consider the various ways to save a user's work and reload a saved file for later use and work on implementing that feature.

Health and Interfaces

I'm baaaack! 

To update anyone who actually reads this (other than Joe Kider): 

I got sick last weekend and after an evening in the hospital and a week pretty much in bed, I'm actually pretty close to better now. (It was just a virus. The hospital thing was, as my Dad says, "in an abundance of caution.") My thinking is, my computer decided to crash, and once it was fixed my body followed suit, each setting me back at least a week. That will mean a little readjusting when it comes to procedures to reach my goals and deadlines originally outlined in my design document, but since I'm still pretty excited about the prospect of sitting and standing, I'm figuring the time will come to handle that. That time might be tomorrow.

In the meantime, I'm reading up on Processing so that I can add things like check-boxes and images and scroll arrows to the tree display. It'd be a shame to do all this planning and coding and then neglect to write the part that allows the users to actually see and interact with all the new features. 

Processing Tutorial Notes (via links): 

Standard Image Loading: 

http://www.processing.org/learning/basics/loaddisplayimage.html

Loading Images from the Web: http://www.processing.org/learning/basics/loadingimages.html

Request Image: Loads in a separate thread so the program doesn't freeze while loading large images

http://www.processing.org/learning/basics/requestimage.html

http://www.processing.org/reference/requestImage_.html

Check Boxes: 

http://mobile.processing.org/reference/reference.php?name=PCheckBox

Custom UI Components:

http://mobile.processing.org/wiki/index.php?n=Tutorials.CreatingACustomUIComponent

Additionally, Interfascia is a library for Processing that handles user interaction, and it looks simple enough that it may be worth using. 

Up and Running

I've got a brand new 500GB hard drive, a reinstalled operating system, and a heck of a lot of catching up to do. Yesterday's project was to get Eclipse, Processing (UI), and Subclipse (SVN) installed so that I can run PhyloWidget and my own code as they were before the crash happened last Sat night (1.5 weeks ago). So that was an accomplishment because I had to remember how I'd installed everything in the past couple months, but I got it working. 

Today's issues are figuring out why my common name search is buggy and then proceeding onto dealing with searching multiple databases for images. At the same time I'll also need to make inroads into writing UI elements for each of these features and then integrating the code and the UI stuff into the PhyloWidget program.

The buggy-ness of the common name search refers to the fact that when I post data to the ubio.com search form and get the resulting HTML page, the page I get is not the same as the page I see when I do the same search in my browser. My code receives the HTML for an "Advance Search" page, rather than a results page. I've had my program print out the URL it's requesting, and it's the same as the results page URL in my browser, yet the HTML is for an advance search page. Totally strange. Any ideas, anyone? 

I'm also concerned about the SVN process, because whenever I want to update to a new revision of PhyloWidget, Eclipse overwrites the one I'm currently using. If I make changes and start including my own code, I want to be able to download revisions without deleting my own work. If you know anything about how to manage this, please share. 

Glad to be back on track, but it's an uphill battle as always!

"Technical Difficulties"

My computer has died... well it's not quite pronounced dead but it's in critical condition and beyond warranty. I'm in the process of rescuing my files off the hard-drive, and tomorrow I'll make some calls and see if someone can take a look at it.
But for the meantime, if it appears that I am not making due progress, this is why. I hope to have this resolved quite shortly, but since I have no idea what the problem is, all I can do is wait and see.

Common Name Search Begins

I am working on a portion of code which will take the scientific names of each organism in a given tree and search for the corresponding common name of each one. I am doing the search using the database at ubio.org. I can currently post a search term to the form and retrieve the resulting HTML page. I can parse that page to determine whether it is the page for a specific organism or whether the program has found multiple possible matches and is asking for more information. 

The current issue is that the HTML I get back from the initial search is not the same as the HTML I get when I do the search in a browser and look at the resulting source code. That's definitely a sizeable issue. The URL is the same, but one's looking for more information and the other is an "Advanced Search" page that I can't even navigate to if I try in my browser. 

I at least have my algorithm written out for how I intend to parse the HTML in order to find the common name of the organism (once I figure out how to get to the correct HTML).

Enter search term into ubio.org. Get resulting page

  

     after text string "Scientific Match" search for "a href ='" string

   save the next text till "'" as a link string

   save a substring of the text after the next > and before <

   compare that to the search term. 

   If it's a match

   follow the saved link 

on the resulting page: 

Find the text "name = 'Common name'"

  After that point, find "namebankID"

  Save a substring of the text after the next > and before <

   return the substring as the common name 

Else 

  repeat by searching for next "a href = '" string

  Stop when you've reached ". 

  Return either the search term or a null string as the common name.

  

Alpha Preparation

I spoke with Val to outline the four specific tasks I’ll need to complete in order to allow the user to input an organism’s name and end up with a tree complete with images and both the common and scientific names for each organism. I divided up the major coding tasks and estimated how long it would take me to do each and which order it would make the most sense, and I used that information to revise my Gantt chart.

I’ve spent the rest of this week revising and updating my design document to reflect what I’ve accomplished and my goals for the rest of the semester.

I also contacted Anne Olsen at The National Biological Information Infrastructure (NBII) to ask about how to search the NBII image database, at the recommendation of Greg Riccardi from Morphbank. 

I'll be meeting with Joe on Thursday morning and Val on Thursday afternoon to review my progress and updated design document. 

Revised Gantt Chart

I am in the process of revising my design document to reflect what I've done up to this point and detail my goals for the rest of the semester. I've already done so for my Gantt chart, so here's a taste of what's going to be in the design doc. 

I wish there were a way to make the image bigger, since I realize it's hard to read the text. Sorry about that. If Joe could add a link to each of our Gantt charts on the Final Projects page for our class website, I could link you to that. What do you think, Joe? 

For now let it suffice to say that I divided up the major coding tasks and estimated how long it would take me to do each and which order it would make the most sense. Those are the tasks represented in blue. Also, the random break in the middle of April is Passover. 

Learning to SVN

Greg, author of PhyloWidget,  is currently updating some of the old code. I wanted to find a way to work on the newest version without having to manually update each file myself based on his changes. The code is posted on GoogleCode, and in order to update my own local version Greg said I needed to use an SVN (Subversion) client. I had no idea what that meant, but I found out that there's an open-source plug-in for Eclipse called Subclipse that does exactly what I need, all within Eclipse. It took more tries than it should have to get it installed and working correctly, but I'm finally able to download the latest code from Google and run the application. Relatively small task, yes, but it took a lot of investigating and troubleshooting, and it's at least a significant accomplishment that I figured most of it out on my own.

For my own reference: 

http://subclipse.tigris.org/servlets/ProjectProcess?pageID=p4wYuA

is the official Subclipse site. 

http://www.ibm.com/developerworks/opensource/library/os-ecl-subversion/

is the site that was most helpful in describing how Subversion and Subclipse work.

http://code.google.com/p/phylowidget/source/checkout

is the GoogleCode website where the most recent code is available.

URL Interactions

Today's task was to create a program where I could test connecting to and interacting with a website. The site I am using is called Morphbank, which is a searchable database containing images of organisms for use by scientists.  

To get myself up to speed about Java and interacting with the internet, I read the networking chapter of "Core Java 2, Volume 2" Using their examples, I was able to create a program to connect with the Morphbank website and read in the HTML of the webpage I wanted. Next, since I'll actually want to input search data, I read through the HTML to find the necessary parameters and wrote a test function to post search data into the online form and return the result page. I filtered through the result HTML to find the lines containing image references and printed those to the screen. The next step will be harvesting those images and including them within the PhyloWidget program. I'll have to learn a little more before I understand exactly how to do that, but coming from absolutely no web-interaction experience and very little knowledge of HTML, I'm pretty proud of what I got done today. 

In other news, I'm in the middle of trying to figure out how to download the latest PhyloWidget code using a SVN client, but so far it hasn't worked. Even when it does, the code is read-only in that format, which won't be so useful to me since I'm hoping to modify it. I need to figure out the best way to update my code and then go ahead and do that. 

In general, I'm glad to say that I'm pretty well on target for next week's Alpha Review. I've made the decision to build on the existing PhyloWidget code in Java using the Processing UI libraries. I've made a sizable dent in understanding how all the code fits together and where I will be interacting with it most, like where to interrupt to cause label name changes, for example. I've refreshed my memory about Java enough to start writing successful programs, and I've learned how to use Eclipse. I've got some details to work out about my ultimate design goals and algorithms before submitting the revised design document, but I think I'm reasonably well on my way to figuring those things out, too.

Label Name Interception

This week I've gotten my hands dirty sifting through lots of code to find both the parser and renderer portions of PhyloWidget. 

The parser reads in the newick strings, and it is found in org.phylowidget.tree.TreeIO.java

public static RootedTree parseNewickString(RootedTree tree, String s)

parses through all the notation, handles determining the different levels of the tree and the parent-child relationships. 

When it's created a string of text it considers the label, it calls:

PhyloNode curNode = newNode(tree, curLabel, nhx, poorMans);

static PhyloNode newNode(RootedTree t, String s, boolean useNhx, boolean poorMan)

...deals with NHX annotation stuff, then...

s = parseNexusLabel(s);

(removes single quotes and replaces underscores with spaces)

t.addVertex(v);

t.setLabel(v, s);

(self explanatory)

So that was pretty exciting, because it allowed me to understand how labels are created and stored in tree nodes, which will be useful since I plan to be able to intercept those labels before render time and change them. In practice, the idea is to display common names in place of scientific names for the organisms in the tree. 

Then I went on to find the renderer. The class LabelRender is found inside NodeRenderer.java. In the method render(), we have

canvas.text(tree.getLabel(n), offX - curTextSize / 3 - s, offY - s - curTextSize / 3);

n is a PhyloNode and tree is a RootedTree so getLabel(vertex) is in RootedTree.java

that calls

vertex.getLabel() which is in PhyloNode.java (extends CachedVertex which extends DefaultVertex)

It is at this point that I intercept the label and change the return value so that the rendered label name is different than the stored one. I created a wrapper class for HashMap called NameLookup, which for testing purposes just stores mappings from each capital letter of the alphabet to the corresponding number, from 1 to 26. 

I created a NameLookup object called map in 

PhyloNode.java and updated the getLabel() method to look up the value in the map and return the resulting string (in this case a number). Any label that is mapped to null in the NameLookup map is changed to the string "Hi Val". The replacement label string gets passed along all the way back to the render. 

For the tree:((A,B),(C,D),(E,F,G,H,'*'),I,J); we end up with the following image

Another way I can do this is store the common names as NHX annotations on the tree, so that I don't have to constantly keep looking up the same names for each update render. I don't know much about NHX, but I'll figure it out if we decide that's the best way. Alternatively, I can just make a "common label" variable in PhyloNode objects and only do the lookup once, when the node is constructed. 

In other news, Val is working on getting me access to image websites so that I can integrate images of the organisms in to the display options. Gotta learn how to interact with web-based applications from within Java, but I took at look at some of the URL and HTML files in the API and it doesn't look too painful. 

New York Times

My friend forwarded me a link to the New York Times article discussing exactly what I'm working on - Tree of Life visualization (and the inherent difficulties involved). Check it out! 

Eclipse/Standalone: One Step Closer

Standalone version: 

I changed the permissions on the .jar files in the lib directory, and the result is a new set of errors: 

lib/freeloader.jar: line 1: PK: command not found

lib/freeloader.jar: line 2: ?h79: command not found

lib/freeloader.jar: line 3: syntax error near unexpected token `)'

K-*??ϳR0?3??r?Cq,HL?HU?%?A??E??%?)?N? ??z?ƺI?'MANIFEST.MF?M??LK-.?

./Phylowidget Full.sh: line 1: lib/itext.jar: cannot execute binary file

./Phylowidget Full.sh: line 1: lib/pdf.jar: cannot execute binary file

lib/phylowidget.jar: line 1: PK: command not found

lib/phylowidget.jar: line 2: ??h9: command not found

lib/phylowidget.jar: line 3: syntax error near unexpected token `)'

?I??'ϳR0?3??r?Cq,HL?HU?%?A??E??%?)?N?@?@TA-INF/MANIFEST.MF?M??LK-.?

Can we call that progress? Maybe. Anyone know what those might mean? 

Source Code: 

I successfully got Eclipse to recognize the source code in the folder I downloaded, and after downloading Processing, importing it as a library, and adding it to the build path, I now only have 2 errors. They have to do the following import statement: 

import com.lowagie.text.Document;

Anyone know what that might be? There's no "com" directory in my source code folder, so I'm not sure where I'm supposed to find that library. I e-mailed Greg Jordan, the author of the code, to see if he could point me in the right direction.

Closer, for sure, but not there yet. Once I get it running, my task is to make at least one visible change to the code, to prove that it's easy enough to work with. Val suggested I try to change the font, size, or color of the leaf node labels on the tree. If I can figure out how to do that with relatively little frustration, then the code is easy enough to navigate and worth building upon. We shall see about all that once I get the thing running.