“… do you always evaluated system in your daily job …”
Basically, almost everyday, I do a lot of evaluation of new system that were to be implemented and streamline the working process in an organsation. It is my nature of my job to be specialised in this process, as that is actually my first degree built on. Specialising in eBusiness Management, where project management experience is a key factor to my interest.
One of the systems, I have evaluated Information Retrieval System (IRS). Basically, most of the evaluation process are almost similiar, but depend of certain factors due to it environment. As this case, IRS are basically evaluated and considered based on 6 criteria. They are:
- Coverage of the collection
- Time lag taken to do the search request
- The form of presentation when the search request is obtain and present
- The effort involved building this system
- The recall of the system (ability to present all the relavant items)
- The precision of the system (ability to present only relavant items)
And, follow by the 5 steps of evaluation process.
- Planning the scope of evaluation
- Designing the evaluation programme
- Execution of the evaluation
- Analysis and interpretation of results
- Modifying the systems in the light of evaluation results
In any system evaluation, there are two major paradigms of IRS Evalution. (i) system oriented and (ii) user studies approach. The difference between system oriented and user studies appoarch are user studies focus on real users. They are analysis more through the qualitative methods and measures. Of course, there are a couple of issues in using the user studies approach in IRS.
- Generalisation – focus on specific user population
- Utility – difficult to see the relationship between the results of user studies and IRS design & evaluation.
- Conceptually – difficult to compare and synthesis results from user studies due to different data collection.
While in system orientated, they are:
- Generalisability
- Validity & Relaibility
- Usefulness
- Conceptual
In general, these are the above factors, I would like to share … For best results, it is better to check with Software Development Life Cycle Methodologies.
Possibly Related Posts:
- Building The Sphere of Movers & Shakers (FB-Malay Community)
- Should not be a librarian anymore
- What’s wrong being a Normal (Academic) student
- Postings in Facebook
- The White House – Blog Post – Taking on Education


Facebook
LinkedIn
YouTube
Twitter
Disqus
Delicious
Stumble Upon
Digg
