As mentioned in the syllabus, the annotated bibliography consists of two main parts: a summarization of the assigned paper, as well as an analysis portion that gives comments, concerns, questions raised, and evaluations of the paper. Note that there is no exact format required; the summarization and analysis maybe interspersed. However, use plain prose, not bullets or numbered lists in your bibliography. The audience for your annotated bibliography should be the typical CSS graduate student who may not have read the paper. Thus, you should provide brief one sentence descriptions of any terms that you use that may not be understandable to the typical CSS grad student. You will have to use your own judgement as to what terms people will generally know about, and what terms need to be defined.
For the summarization portion of the annotated bibliography, aim to provide a description of the results and/or the purpose of the paper, as well as a brief overview of the ideas, implemented algorithms, case studies, and/or logical arguments on which the paper is based. Since this is a summary, not every single idea need be presented. As a guideline, aim to present several of the main ideas in the paper in just enough detail that they are understandable by someone who has not read the paper.
Explanation of the ideas in the paper is more important than defining actual terms. Take the term UDDI for example. UDDI is a protocol for describing and storing directory information for web services. Instead of saying "this paper studies several different ways to implement UDDI" and then defining UDDI, you could instead say "this paper studies several different methods for storing directory information for web services" and not mention UDDI at all.
For the analysis portion of the project, you are trying to judge the quality of the paper, as well as talk about the paper in a broader context than just the what the authors presented in the paper.
For the purposes of evaluating the paper, consider the following questions:
For papers which you do not understand fully, comment on which concepts and/or ideas you did not understand, why you did not undertand them (if you can pinpoint why), and what sort of background you think you would need in order to understand the paper fully.
The annotated bibliographies are partly for you to reference in the future. Thus, you may also wish to keep additional information with your annotations, such as the date when you read the bib entry, where this reference was first cited (i.e. how you heard about it in the first place), other works in your bibliography that it cites, as well as similarities to other works in your bibliography. You also may want to list some keywords associated with the entry for later searching and retrieval.
You do not have to answer all of the questions in order to get full credit on the analaysis/reflection portion of the annotated bibliography. You only need to show a significant amount of thought and clarity of reasoning in the comments that you provide.
Annotated Bibliographies will be given two integer scores between 0 and 4. One score is for the analysis portion reflecting your comments, reactions, questions on the paper. The other score is for everything else, including the summary, having complete bibliographic information, grammar, and good writing style. The overall score will be the average of the two scores. I expect the average score to be a 3. Thus, do not expect to get a 4 unless you have perfect grammar, good writing style, excellent summaries, and a large amount of good, quality comments on the paper. For writing style guidelines, see this page.
Here is an example of an annotated bibliography receiving full credit. The paper is here. The annotated bibliography is here. It was written by Josh Tenenberg and then slightly edited by Ed Hong.
An example of some entries from Josh Tenenberg's annotated bibliography is located here. Note that his bibliographies are fairly light on the summary of the papers but they do an extremely good job at the analysis/reaction/comments. Some of these bibliographies would not receive full credit on the summary portion.
The assigned essays will be graded on an integer scale between 0 and 4. The levels of good writing presented on the writing hints page can be mapped to the integer score roughly as follows:
| Writing Level | Integer Score |
|---|---|
| excellent | 4 |
| very good | 3 |
| good | 2 or 3, depending on quality |
| average | 2 |
| below average | 1 |
| poor | 0 |
| failing | 0 |
Your comments should be given in well-written prose (do not provide a list of bullet points). The critiques are expected to be fairly short, at most a few paragraphs, although I will not penalize length. They will be graded on a 0 to 2 scale, with the following meanings:
| Integer Score | Meaning |
|---|---|
| 2 | comments appear valid, reasonable amount of effort put in |
| 1 | some comments do not seem valid, or questionable amount of effort put into critique |
| 0 | not turned in, or appears like very little effort put into critique |
The project topic description is meant to give a background overview to your chosen area and to describe what you plan to learn about in the future. In overviewing your area, you need to address several types of questions:
In terms of the grading scheme, the integer scores are given according to the writing level table given above (in the guidelines for assigned essays). However, each required component not adequately addressed may result in a penalty of up to one point.
The requirements for the content include a well-defined scope for the proposal, as well as an evaluation of merit of the proposal. To help define your scope, you should think about the end products for your project. Are you going to perform case-studies and pass out surveys and tabulate them? Are you writing a research report based on reading about different ways people have implemented a certain concept? Are you doing some implmentation work and creating software to solve some particular problem? A well-defined scope means that the nature of the project you wish to do is clear, and it is possible to roughly estimate the amount of work necessary to complete your project. Thus, you should also answer questions like: How do you know when you are done with your research reading or with your implementation? If you are doing an implementation, are you starting from scratch, or building off of some existing project? How will you evaluate the success of your project after you have completed it? The merit of the proposal should address both technical merit (such as amount of new CS concepts learned and applied, significance and novelty of the proposal, describing why the problem is technically difficult), as well broader context benefits.
You are trying to convince the reader that this proposal is significant in its impact to the scientific community. Your proposal should also be of a scope that could be expected to be accomplished in a capstone project. Scores of 4 will be given to proposals of the proper scope that clearly have merit. Scores of 3 will be given to proposals of the proper scope which seems plausibly significant (in terms of merit) but whose arguments for merit are not completely convincing or are somehow deficient. Scores of 2 and below will be given to proposals without proper scope (or ill-defined scope) or to projects whose scope is clear but whose merit is lacking or not evident. In addition to the scores just mentioned, mediocre writing quality or lack of quality references may result in a one point penalty. Note that having quality references is very important; however, the quality of your references will be judged primarily in the grade for the project annotated bibliographies.