Workshop notes 2007 july 12

Agenda Items carried over from Wednesday

8. Revisiting other agenda items suggested Monday pm to decide what to do

• Env ontology discussion. Suzi Lewis has posted a response re: Michael Ashburners GEO project.

SS reports that Dawn Field is organising a workshop for Env ontology in August. JF:I recall was including lab or experimental env that obi needs. Need to be clear if the lab env will be part of the env. ontology or as part of them working with OBI. Dawn Field is part of OBI CS:We need field isolates e.g. for parasites, we will need what they cover. CS: Where is the delineation between what OBI should add about the environment and what EnvO should do? SS: We should do the experiment and clinical environment, and the "external" environment to EnvO. But, we aren't too bothered, if envO people want to do it.

AA:Susanna to communicate the above and feedback

• Liju, are we going to reserve some time to go over terms in each branch

HP:We will not have time to do that, too many CS:If there are questions about terms, suggest raise them, need to trust the branches. Expect people to go through the branches if they want to. We could set aside problem terms HP:we need to address flagging unclassified terms consistently before we can do that

• How to link to the external ontologies, Bill identified has a way to do that in BIRNLEX. Would be good to have him show that, today

JM:Not sure if you can import a subset of an OWL ontology, one way round is to create a local subset and import that as a 'view'

AA:Allyson emails Bill to ask for a summary of how OBI should address point 10 below.

9. Flagging 'helper classes', e.g. items that are 'unclassified', consistently

We check vs. previous meta data terms we noted some to be added.

Placeholder till something can be referred out to belong in some external ontologies Placeholder for a set of collected terms.

CS:under curation status there are a list of terms

JM:'temporary class' ? AL:want to make a difference between things that will be referred out to other ontologies. we add a new meta data 'awaiting external mapping' These have been added to the wiki by AL. CS:if editors want to not classify terms under an existing class, they should them hanging free under their branch root. BP:would prefer a to put in a holding class as James suggested CS:if we have a consistent temp class that would be fine BP:Should it be Protocol_Application_temporary LF:there's a string search can find anything you want to. . AA:All branches to add branchname_temporary and use it consistently. This class will not need to have any meta data annotation as this doesn't merit it. AA:Ask Trish to send the code for making the figure Add a new item to SOP for reasoning Identified people able to support reasoning : Daniel, James, Alan, Gilberto, Allyson

DS: The use of only one 'branchname_temporary' helper class per branch might not be enough to cover our branches needs. It will force us to overload this with meanings, e.g. as the list above indicates, people might want to have some more specialised helper classes, e.g. a '_refactor_as_relation' 'awaiting external mapping' , '_submit_to_xy_branch' or whatever. It would be nice to give Editors more freedom here. Also the required helper classes are most probably branch specific as well. I would suggest to have these on the branch top level/branch root and indicate their status with the underscore prefix. The uinderscore prefix serves as search string(startswith) and also renders helper classes easy visible in the hierarchy. The _underscore is recognized in the hierarchy more easily as when using just a normal looking names like branchname_temporary ("_temporary" postfix). We need to identify needed helper classes (on a branch and on a general level), and what info do we want to see immediately in the hierarchy (and use for browsing), and what info we want to keep via annotation properties (the ones we want to do more fancy querying on). Then later we can review all of these and probably re-formulate as part of some metadata standard.

10. Referencing, mapping to, or importing (subsets of) external ontologies from/into OBI

Bill joins by phone.

Alan provided a worked example in SVN directory, added to the worked example from yesterday OWL file. Went to protege editor, took class out of PATO full ontology, for class mobility. I wanted to import the label, the definition by editing the XML, I put that in the comment. Logically snip a part of PATO, paste into OBI OWL file. If you can do inside protege not sure. Need to tell it about the namespace. Smoothest way needs to be worked in practice BB:comment you've done this manually, Protege does learn. Once you have a URI to reference you can get in some of what's at the URI. AR:someone might be interested in a protege plugin GF:This was one of the driving forces for NCICB EVS to start moving to OWL, we wanted to consume the world, and not reinvent it. Two ways to do it. Depends on 1. if you want to model vs PATO and you want to include you can import all ontology. Advantage you can reason with PATO namespace. In some cases don't want whole thing, you just want to reference a concept, do in this way. This is the question if we include the other resources, we don't need to publish all namespaces depending on. If we want all, we can get all. AR:there are cases where the ontology is too big, or may have mistakes JF:definition of moblity is not aristot. would we not need the parent term? GFIf you want the whole namespace yes, you need it. EVS will do that, in this case if we add a node, the classifier ignores it even if it was DL defined in PATO. BB:other option is a script that generates OBI for different needs, we could have cases where we keep it minimal and others were we bring in all names spaces. RS:Example is NCBI taxonomy, there will be limitations as i shuge AR:We could have a meta data property where we add external definition, if we need a local definition we could add it in OBI definition BB:there's a synch and maintenance issues AR:that can be scripted. BB:only problem we've (Birnlex) had is that it's easy to script hard to do as protege plugins AR:We need a policy that when other ontologies are up to BFO stds then they will come in AL:if we don't have a plug in we could add to the release steps BP:note that we are providing infrastructure to link to other ontologies. We need agreement from others where we bring on other things. We should have agreement that SO lives under information object or we will be revising their worlds. BB:we need an OBO wide meeting to discuss that. HP:we could attempt to map OBO vocabs to the OBI structure and would help us evaluate OBI structure and would be a good basis for discussion

Add link to file

%Quality OBI class
  %mobility (pato name space, imported into OWL file'

Bill presents on Birnlex BFO relation ontology, notion of upper bio layer, I think there is a point to defining something called OBO foundry core, BFO, bridge files from Chris Mungall map relations ontology RFO 1 mapped to BFO and PATO into a single OWL file. Is BFO plus RO and PATO. Some issues about Protege not knowing how to render things so some looks strange, as was manually imported. AL:Does protege 4 do a better job BB:for some things, also OWL 1.1 AR:Protege 4 is that we could ask for a plugin for that AL:could you do that? Also have some other bugs BB:we need to do more complex entities for OBO, and do I want all of PATO- we BIRNLEX decided to get it all, and we didn't want to deal with automating getting parts of protege, . PATO doesn't add too much. For a while we had used OWL docs for people to do an html version and once added pato we can't generate OWL docs any more. We will use bioportal despite problems. BB:we have used a lot of annotation properties in working wit this file. All NCBI tax is a huge file and a lot of meta data in taxonomy. You don't want it as is, we decided to look at the organisms we needed. RS:General question, when we bring a namespace, is there only one place that it sits in OBI. Is that true for all OBO foundries BB:IT SHOULD BE, not always true. In tax it breaks when you get to strains for e.g. and that needs to tie into taxonomy e.g. Mus musculus from strain for e.g. Protege UI is bad for large h'archies. We're dealing with that as a UI issue as NCBI tax so big. We use annotation properties to mark taxons that we need in Birnlex.

BB: In the taxonomy file, if you want all of NCBI taxonomy it is a huge file with huge classes and a huge amount of metadata. Also, it isn't an ontology so you don't want to bring it into the ontology anyway. He just imported what they need. You can build what you need when you need it. they also had to create their own classes in this hierarchy, e.g. hybrid_strains_of_mus_musculus_and_mus_domesticus. NCBI Taxonomy "transliterated" into OWL using the BIRNLex conventions. Done via a script. owl:imports, OWL ObjectProperties, CO-OWL, E-Connections are also available to perform inter-ontology linking. DS: Here is a link: C-OWL: contextualizing ontologies: http://www.kw.man.ac.uk/doc/wp25cowl-040324.ppt CS: We're co-developing with both PATO and BFO, so we shouldn't try to import all of PATO immediately, but when we publish it must be a serious consideration. DS: Regarding import and mapping we definitely all should read this: http://obofoundry.org/wiki/index.php?title=Mappings&oldid=1372. Import and referencing is a huge issues on itself and we need a task force that have the time to intensively work on this issue.

BB: what you saw is from the "pending" release of BIRNLex. The current public release was rushed out before it was really ready in order to get it into the BioPortal. As I mentioned, that still hasn't happened. All of the modules in the next iteration of BIRNLex will be guaranteed to be OWL-DL, consistent (and in some versions, satisfiable). Anyway - here's the URL's the files I showed:

1. OBO Foundry core files http://purl.org/nbirn/birnlex/obo-foundry/obo-foundry-core-full-import.owl This file imports several other files we've created - in addition to the public files from BFO, RO, PATO, and Chris M.
2. There is also a work-in-progress version that doesn't import ALL of PATO - but includes additional reference to guarantee the result is passed as OWL-DL species and consistent by Pellet: http://purl.org/nbirn/birnlex/obo-foundry/obo-foundry-core-no-full-pato.owl
3. Here's the files with the organism taxonomy: http://purl.org/nbirn/birnlex/1.2.3/BIRNLex-OrganismalTaxonomy.owl

Summary Bill has a local use ontology with RFO, OBI, PATO,NCBI taxonomy where all of these are imported into his local file. This 2 years work. He chose to do that as it was more complex and technically difficult to take parts than live the problems of having huge in built taxonomies. Bill recommends that if in doubt get all. Protege 4 offers more options and a plug in could be created. All OBO foundry ontologies should be located in one place in OBO. Take home, modularize and lean on the reasoner.

GF:at some point might be good to work out whether we want a set of external resources that we want to import and which we want to reference and make a recommendation for branch development

BB:Susanna's workflow has these issues. Susanna agrees useful for biomaterial branch, she described the manual process. For automation e.g. with NCBI tax, find ids for a subset to get what need is use case. Need to be careful about if you want external h'archy as well.

CS:we have a special relationship with PATO, BFO and we need to decide after discussion if we want to import them AR:I would look at again after 6 months DS by Skype:interested and wants to work on this

Conclusion: we will have terms sent out to relevant ontologies until the point where we need to get all of PATO in and then we need to be aware of PATO's internal structure.

Citations

• link to C-OWL: contextualizing ontologies (Bill was talking about it): [[1]]

• Obo foundry page:

[[2]]

11. SOP for ontology development, PDF Trish sent

RS:we have a need here for OCI to work on all OCI terms and then bounce these to OBI branches. Will also need a model for the env. ontology. So OBI branch editors would validate a term from the community. AR:we need to allow users not to figure out parent class LF:they should feel bad if they don't give a definition AR:some field are more important than others, parent class is not so important LF: How would relations fit into the SOP? Do we need to make a relation workflow? AL & LF: We can use both as long as we change "term" to "term/relation" RS: Minor and Major releases, with the use-case testing only done for major releases. GF: Do we want to publish both asserted and inferred ontologies? AL & AR: Yes. AR: Some editor programs would only save the class hierarchy of the inferred ontology, so we may want to think about that later on. AL:needs a few changes as we will not merge files as we expected previously. Had asked to add a bit about reasoning, and Trish did that. AR:Need to specify which reasoner, no unspecified OWL classes etc. Need the OWL-DL version is checked. Liju need to check the asserted h'archy to verify the modelling. AR:need to save and need branches to check their inferred version is correct

Problem that there's a shortage of OWL literate developers to do reasoning etc. How does that effect the workflow.

GF:we do classification routinely in our group. I would not mind doing that and I am set up for it. Would be doing an import and passing around. AR:Chris Mungall has set up for OBO Pellet reasoning to produce a daily report we could also do that SS:need a pool of people with this expertise, GF, Daniel S, James, Alan, Alison GF:relations between terms is pretty far down. This will be an iterative process, some relations will be merged, needs a loop to represent that. Most issues will be on file merging. RS:suggests major and minor release pattern

12. suggestion to do some work on experiment design, goal, etc beginning to end example suggested by Kevin and Alan

• OBIAnnotationUseCaseExamples As emailed from Kevin Clancy
• Google Spreadsheet version
• Alan's OWL workup of this example

Long Discussion whether to model an instance of a hypothesis, or the components and relns between obi classes e.g. object, subject are related to hypothesis. Bill suggests we take a look at [[3]] and EXPO was also referenced as having addressed this before. Modeling it has value as we can extrapolate to other cases. Problem about modelling an instantiated hypothesis, as is statements are expected to be true, hypothesis may not be true and this presents a modelling problem. Same is true in cases where record exp. knowledge over time, and are represented as true but may not be. There's a scope question, should instantiated hypotheses be in OBI, is this in scope for us.

We are treading a line between knowledge base and high level modeling, modeling both here as an exercise is seen to have value as we will identify holes in the ontology and

We proceed with the modeling in OBI, and also summarized on the spreadsheet attached above.

Discussion on relevance of reification. BP: What we've realized is that there is a strong need for certain communities to be able to make contradictory statements. These statements need to be structured in the same format as standard OBI format, but it should be clear that these are conclusions/hypothesis, which means that they will not agree with each other and they will not be "consensus biological truth." We've discussed many possible technical implementations of that (reification). AR: Every conclusion must, at a minimum, conform to the consensus reality. Alternative is not make a statement. CS: If you just say conclusions have parts, provided enough info for comprehension, no assertions made across them, conclusion subject 'aspirin' etc, subject 'mouse' no statement about the detail of the hypothesis and this is enough info to say something useful. We can do both of these things if needed. I could identify investigations involving aspirins but I couldn't find cases that had drawn a conclusion. CS:we both have the same use cases, papers and studies, and we will get a text statement, biologists won't be able to do this.

GF:interested the ability to annotate an expt, if you can reason if is annotated corrected - this use case should be subsumed by the one above.

AA. Need to come up with a reference implementation proposal, will take time Current state of the art is we can tag text with hypothesis and conclusions are facts which are consistent with consensus biology as represented by the OBO foundry world.

CC:both hypotheses and conclusions are roles of sentences, they are defined by inferential role and same sentence could be represented by the diff roles. General Agreement for this.

AR:Using role loosely, may not be in OBI, could do this in an implementation. CC:think we should have these roles in OBI in roles branch BP:we can CC:I am playing devils advocate, Barry would have an issue with this. CS:should hypotheses by narratuibe objects, should we have a dift type of object that has a role? We may want to define more specifically BP:we said hypothesis and conclusions that are descriptions of reality CC:declarative sentences are desc. of reality BB:disagrees, 'Homer chases a donut' is not about reality CS:Barry will say that hypotheses etc are roles, BS will not like this BP:Barry did sign off on these, we tired him out RS:hypothesis are here as we did not know where to put them, now we can move if needed CS:This branch is renewed and we will address this GF:I asked KC if he had a suggestion of moving on. He suggested nice if once I annotate an experiment I can validate hypothesis and conclusion are the same, so an instance can be both if the experiment is successful AR:in principle, but OBO things are time dependent which complicates things GF:thought we wanted to id the set of conditions asserted on a class, so when you define an instance of that class that the classifier will work correctly Discussion on whether we want to do this now GF:I heard that this will take 3 months, if it will then we are doing something wrong. We can model and define some classes to make hypothesis a defined class in the ontology.

AA: AR suggests that GF defines hypothesis. GF my background is in a certain type of biology, we should have more coverage, to me this is a good place to deal with hypothesis. BB:PAPO is back end for writing observations for which to build a hypothesis. BP:look at do it briefly for Kevin's example, unless there are other things. We move to point 10.

Here's a few of those links Alan & Bill & others had mentioned: SWAN - http://www.mind-informatics.org:8081/swan/ Phenotype Statements using PATO - W3C HCLS example - http://esw.w3.org/topic/HCLS/OntologyTaskForce/OboPhenotypeSyntaxExperiment Nigah Shah's (NCBO) HyBrow - http://www.hybrow.org/ EXPO - http://sourceforge.net/projects/expo/ and associated Robot Scientist project - http://www.aber.ac.uk/compsci/Research/bio/robotsci/

12b. Discussion continued on Friday

Had narrative object statements as part of the investigation We noted also that we needed to move to consensus view of biology. Need a link between these and facts. Alan had different versions of that.

Aspirin has quality anti-inflammatory BP:IS quality short for information processes CC:Should be in BFO BS doesn't see the use of having qualities of processes, we should have that. AL:lets make statements of all things we need JF:all animals are not equal BP:this is not a hypothesis, it;s an observation not equal in all mice - need a quality of moderate - is in pato would anti-inflammatory is a quality about the aspirin BP:can we define function in relation to quality?

13. Competency questions, testing and evaluation

Daniel and James have volunteered for this. This requires input from everyone in terms of developing use cases along with competency questions. Need also requirements.

JM:When will we be able to know when OBI is done. What does a stable version mean.? GF:If we volunteer use cases and comp. questions if it works for these use cases then when we can answer them, this is a stable release. JM:E.g. of competency question, is what do I want it to do, e.h. where will hypothesis go. What type of people will use this, how will they use it. Stuff that came from the programme managers GF:Do you have any concrete plans - questionnaires? Or are free form use cases OK? JM:We do have some ideas, we have both done it before. DS wanted to know about conforming to certain grammar, please use instances variably. HP:suggest that the more you constrain the process then you will get a better result PRS:Kevin has produced something useful, we should all take a lead from what Kevin did, that's really a good example. RS:a use case template would be great, we have clinical use cases for OCI. If we had a template we would provide them BB. We would make a great contribution to the field if we could do this. BP:What kevin sent is important, we need high level use cases JM:need a range of use cases at various levels AR:one test we could do would be to ask a series of yes/no questions you will know where to put your term in the ontology PRS:also text mining was mentioned BB:is a high level use case category PRS:This a.m. I was expecting to have an idea to use hypothesis, and I was looking at how to put constraints on that, I view that as a use case. Maybe I don't understand what OWL can do or my usage is reconcilable with what we are doing here JM:A question can the ontology do this, that is a competency testing. If you can answer yes to all then are finished AR:nonsense detection is another type of CT, we should be able to add to reasoner and have it complain, e.g. is some class an experiment and a quality, PI is a mouse. You design test s to check that coverage is enough to do this. RS:first official release of OBI needs such validation PRS:in the absence of a list of competency questions, we don't know if we are doing it correctly, this worries me JM:scope and requirements should be the first thing that should do, but don't worry we can work from here CC:Suggestion: as far as I know, what we are doing is another knowledge representation style, we can ask what competency questions should any KR style ask? BP:And then what differentiates the ontology style of OBI within range of KR styles JM:Agree with CC, we will need this 'pitch' and this was clear on Monday when the programme officers asked how we would sell it to people. We should be doing this all the time. AR:A representative data set of interest and we do the encoding and add to science commons triple store LF: from http://protege.stanford.edu/publications/ontology_development/ontology101-noy-mcguinness.htmlCompetency questions:

"One of the ways to determine the scope of the ontology is to sketch a list of questions that a knowledge base based on the ontology should be able to answer, competency questions (Gruninger and Fox 1995). These questions will serve as the litmus test later: Does the ontology contain enough information to answer these types of questions? Do the answers require a particular level of detail or representation of a particular area? These competency questions are just a sketch and do not need to be exhaustive. In the wine and food domain, the following are the possible competency questions:

• Which wine characteristics should I consider when choosing a wine?
• Is Bordeaux a red or white wine?
• Does Cabernet Sauvignon go well with seafood?
• What is the best choice of wine for grilled meat?
• Which characteristics of a wine affect its appropriateness for a dish?
• Does a bouquet or body of a specific wine change with vintage year?
• What were good vintages for Napa Zinfandel?

Judging from this list of questions, the ontology will include the information on various wine characteristics and wine types, vintage years—good and bad ones—classifications of foods that matter for choosing an appropriate wine, recommended combinations of wine and food."

SS: If nedeed see definition and example of competency questions at: http://www.biomedcentral.com/content/pdf/1471-2105-7-267.pdf page 7 (we did this for the nutrigenomics group while back).

AA.Use case templates would be good AA.Worked examples use cases needed AA.List of questions that allow you to place a term in the ontology

14. Structure in general, branch organisation

Lead developer for every branch, pool core of developers discussion prompted this agenda item. We have id a pool of core technical developers. Editor-in-chief became branch facilitator. Idea was to be useful to get branches up and running and to help moderate branch cross communication. I did less of more of former and less of latter. Is the role still useful, my efforts could be best spent as an editor and I have offered to join the digital entity role and I don't want to continue as editor in chief BP:I can imagine now we will have a need for a faciliator more. GF:Second. RS:I expect at some point we need one, defer till then.

Consensus: we temporarily suspended the role of editor in chief until problems arise.

15. Use of mailing lists, traffic etc

Mailing list traffic too high, all important, maybe can filter. One would be voting - wiki voting. Survey monkey is good for organization issues.

HP:suggest that people summarize in emails using bullet points and put it on the wiki if it's long.

AR:suggest that an email editor puts them on the wiki and people get the good stuff that way. Or filter your email properly

BP:One way to do this is to define sub categories of email trackers - term tracker email list, would be great.

JM:When people want to talk a lot, set up a forum or google group and go read that, or use a tracker

AR:all this will work for the people in the room, what about the others

HP:the job of email editor sucks, I personally will never do that, BP agreed.

Options:

AR:Google indexing, perhaps some should not be public. All lists are available including Advisors GF:think advisors mail should be private. CS:in all cases including OBI there are cases when private mails are needed. We have seen this in OBI AL:sugest public coord issues stays open, new one obi-private, these will never be indexed. AR:Don't know if can turn that off in sourceforge. AL:Trish, Allyson, and Daniel S. are project admins on SF AR:Propose all our open mailing lists should be indexed by google. Alan has already put a bug report in. BB:in terms of minutes for calls. You can run a internet relay chat system along with a conference call. Would take the burden of keeping minutes. CS:call costs are 200-225 USD per week. We told the MGED board, this is OK, we might look for a cheaper call in system, this may not be permanent SS:EBI has a number, but there is a local call no, each would pay a local call, MGED would not then pay GF:In MO calls we used NCI, these are non standing telecons, each person must be called by an operator. Skype is not allowed in NIH.

Options

• conference calling is expensive, we may need to find an alternative
• skype is not really an option, too many people, poor quality.
• centra supports VOIP, we could test. Small delay only. Linux unsupported.
• EBI has a system, is not toll free for call in, but Susanna will check if these are local nos for all.

16. Stable set of core OBI or at least terms/def (that can be used asap)

SS:I need a set of terms now, to annotate things that need to be done now. Clean, defined. When will we have that BP:we already trying to be future proof, is there a stable subset HP:Want to keep milestones in sight and we should add these asap age for this AR:suggest put a term in, not sure there's a good parent, add them in quickly into unclassified bin the queue is there, not on a sheet SS:want to have them go through the workflow really. AR:think all terms there will stay, suggest that we do this more aggresively not worry about the h'archy. PRS:is there a risk that we don't do the review we will pollute the h'archy. RS:we need to come up with an aggresive time line for OBI core version. AR:terms and the h'archy equal for you? CS:this is a pipedream. We need to do it right. RS: I want a quality core, that comes before speed JM:Bill mentioned that, you'll get one chance to get it right BP:we never going to know until we apply it. Disagree with JM. We could have an internal stable version BB:I want to be clear, I agree with everyone, those who are using and developing of part of OBI take it on as a liability in the interim as an internal unstable version CS:what does that mean for a public release PRS:want to go back to Kevin's use case today and see if it works, of it doesn't then we have a lot to do today AR:more frequent releases, and qualify stable for each release BP:stable but usable, need an agressive timeline that we acknowledge JM:we are not in complete disagreement, people will complain if it doesn't work

Some agreement for an alpha public version asap

GF:Susanna is looking for more than that, terms and codes, I can go with that HP:I think we are now getting into milestones.

• BB:proposes 2 layer down from OBI ready. Several people agree that this is useful.

BB:even top layers are alpha, but we are closer

Milestone 1

Release in which many terms, not nec. correctly placed, ids won't change, placement will

Milestone 2

Release in which we have 2 levels down from branch stubs and we are happy with this

17. NSF proposal details from Alan

AR: Grant proposal: http://www.nsf.gov/pubs/2007/nsf07565/nsf07565.htm