Taxodiary – Access Insights

Leveraging Your Taxonomy – Part 10 (Taxonomies in SharePoint)

By Melody Smith|April 9th, 2012|Access Insights, Featured, search, Taxonomy|1 Comment

I hope this series on search has been helpful to users and professionals alike. Let’s close with a look at taxonomies in SharePoint. Let’s look at this data flow in another way. We have incoming information; going to dump into a repository. We need to add metadata to that repository. We want to add taxonomy terms. The taxonomy terms all need to be controlled or suggested. So, there’s a backend to do that. Once we have the data in that repository it could be exported to a SQL or a relational database, transactional system, for e-commerce. It might be put into a repository so that the full displays can be done. It might be loaded into a search system and you also might have a presentation layer for display.

Access Innovations, Inc. Creates Taxonomy For Iowa Code, Administrative Code and Acts

By Melody Smith|April 5th, 2012|Access Insights, indexing, News, Term lists|2 Comments

Access Innovations, Inc., a leader in the data management industry, has collaborated with the Iowa Legislative Services Agency to build a customized thesaurus that allows the Iowa Legislature General Assembly to easily access its extensive legal body of existing and proposed laws, bills, acts, and regulations by using controlled, vocabulary-driven indexing in addition to published indexing codes.

Leveraging Your Taxonomy – Part 9

By Melody Smith|April 2nd, 2012|Access Insights, Featured, search, Taxonomy|Comments Off on Leveraging Your Taxonomy – Part 9

As we continue the series on search, we are close to wrapping up with a more in-depth look behind the scenes of database management systems. Let’s take a quick look at behind the scenes. We want to connect the database management system to the thesaurus tool so that we can validate the terms and make sure that they are in good shape and, as people are adding records to the database, if they have any suggestions or candidates, we want to lock those in as well. The thesaurus tool will tell you which terms are actually correct, allow you to add, change, and delete, and otherwise manage the term base. Then the indexing is used to actually suggest indexing terms to records as they are loaded to the database management system. That system can be SharePoint, it could be a content management system, it could be a Documentum or a FileNet, or any other thing you want to use as a repository to manage your data. That is driven by the taxonomy.

Leveraging Your Taxonomy – Part 8

By Melody Smith|March 26th, 2012|Access Insights, Featured, search|Comments Off on Leveraging Your Taxonomy – Part 8

As we continue the series on search and how it works we are looking at file indexes more completely, more specifically complex inverted file indexes. Stemming is the de-pluralization or removing the gerund endings. It is also called lemmatization. Truncation – left and right – are popular parts of search. Right, in some cases, chopping a word off at its end; is pretty easy. Left-hand truncation is hard because if you look at this wild card in the word ‘organization’ which can be spelled with either an ‘s’ or a ‘z’, depending on where you are from, the ‘-ation’ can be chopped off pretty easily but the right part, I have to build an entire index, starting with o, or, org, org, so that I can go through all of those to see where the full extension is. When people do left-hand truncation, it is a lot more expensive. It is a much bigger, additional index.

Leveraging Your Taxonomy – Part 7

By Melody Smith|March 19th, 2012|Access Insights, Featured, search, Taxonomy|Comments Off on Leveraging Your Taxonomy – Part 7

This is the next piece in our series of blog posts on search and how it works. Next let’s look at an inverted file index, let’s pretend that this is the outline of the presentation. I have Define Key Terminology, Thesaurus Tools, Functions, Features, Class, Construction of the Thesaurus etc in the figure below. You can see that the word “Thesaurus” is used three times here. I have a number of other words that you might focus on to see where they are. If I am going to take these and make them into an inverted file, the simple inverted file index is just going to take them and make them into an alphabetic list. So it will sort the high ASCII characters first – the special characters and the numbers – and then it will sort the rest of them alphabetically.

Leveraging Your Taxonomy – Part 6

By Melody Smith|March 12th, 2012|Access Insights, Featured, search, Taxonomy|Comments Off on Leveraging Your Taxonomy – Part 6

This is the next piece in our series of blog posts on search and how it works. Last week we ended with natural language processing and we are picking up this week on automatic language processing or ALP.

Leveraging Your Taxonomy – Part 5

By Melody Smith|March 5th, 2012|Access Insights, Featured, search, semantic|Comments Off on Leveraging Your Taxonomy – Part 5

This is the next piece in our series of blog posts on search and how it works This time we are talking about the importance of relevancy. Relevance is how well a set of returned documents answers the information need, another way of talking about accuracy. But, it’s related to the objective of the search so different user communities can get exactly the same answer from a set of information resources and one will find the set relevant and the other will not. So, there’s a really healthy tension between the user needs and the context available. That’s why a lot of relevance engines do a lot of profiling of the users so that, if I search Google for a particular question, I might get one answer and each of you will search it and get a different answer. That’s because your profile and the things that you’ve clicked on in the past will indicate to Google that your answers should be more in this sphere or more in that sphere. So, relevance is really a confidence factor or a guesstimate on how well this set of documents will answer this particular user’s query.

Leveraging Your Taxonomy – Part 4

By Melody Smith|February 27th, 2012|Access Insights, Featured, search|Comments Off on Leveraging Your Taxonomy – Part 4

As we continue this series on search and how it works, we have to address accuracy. First, how are we are going to measure accuracy?

Leveraging Your Taxonomy – Part 3

By Melody Smith|February 20th, 2012|Access Insights, Featured, search, Taxonomy|Comments Off on Leveraging Your Taxonomy – Part 3

This series of blog posts is exploring how search works. We need to have a basic understanding of search fundamentals in order to know where taxonomies come in. Last week we started talking about search software and today we will continue with that topic. I believe in the data first as you know. Staring in the diagram with your Source Data you can see how the data flows. You need to clean the source data to a uniform format. This is often called the conversion process or the ETL - Extract Transform and Load.

Leveraging Your Taxonomy – Part 2

By Melody Smith|February 13th, 2012|Access Insights, Featured, search, Taxonomy|Comments Off on Leveraging Your Taxonomy – Part 2

This series of blog posts is exploring how search works. We need to have a basic understanding of search fundamentals in order to know where taxonomies come in. Last week we started with the various modules of search. This week we are addressing the search software itself.