In an observational study, we noticed that record-keeping plays a critical role in the overall process of collaborative visual data analysis. Record-keeping involves recording material for later use, ranging from data about the visual analysis processes and visualization states to notes and annotations that externalize user insights, findings, and hypotheses. In our study, co-located teams worked on collaborative visual analytics tasks using large interactive wall and tabletop displays. Part of our findings is a collaborative data analysis framework that encompasses record-keeping as one of the main activities. In this paper, our primary focus is on note-taking activity. Based on our observations, we characterize notes according to their content, scope, and usage, and describe how they fit into a process of collaborative data analysis. We then discuss suggestions to improve the design of note-taking functionality for co-located collaborative visual analytics tools.