#Critlib February 10, 2015: Assessment

#Critlib February 10, 2015: Assessment

  1. As usual, #critlib kicked off with some buzz and introductions
  2. Excited to be hosting #critlib for next hour - assessment w accreditation, program review, and other standards! IL Coord at UIUC
  3. Introductions continued to trickle in while we moved on to Question 1: What types of accreditation, program review, governmental, etc. standards are in play at your institution?
  4. Shall we start #critlib Qs? Q1: What types of accreditation, program review, governmental, etc. standards are in play at your institution?
  5. A1 Lots of outcomes assessment, looming program review #critlib
  6. A1 #critlib WASC accreditation, program review based on strategic plan, periodic curricular revisions (GE and writing recently)
  7. a1 part 2 #critlib day to day discipline specific agencies are important to branch libraries (ASC, NASW, etc.)
  8. #critlib a1: My uni has annual assess reports for all programs, including lib instruction. Uni going up for re-accred from HLC in 2 years.
  9. #critlib a1: Also, state of CO gen ed board revisiting outcomes- going to add info lit requirement, though it'll be based on old standards.
  10. @kevinseeber thanks for adding in govt example #critlib (fortunately, can map from any doc to another!)
  11. @kevinseeber Interesting. I'd look forward to hearing more about that process, esp. as Standards factor into it. #critlib
  12. A1: the obvious one from a student perspective is ALA. ARL. Also a lot of other ones that don't register for me on a day-to-day #critlib
  13. @kshockey04 ALA for the LIS program accred? Not sure what ARL would review? #critlib
  14. We have a lot of different groups assessing and accrediting our programs - too many to name. #critlib
  15. @ibeilin Same! Not exactly that I don't know, but hard to make sense of. #critlib
  16. @beccakatharine It depends on one's subject areas - my areas haven't come up yet. #critlib
  17. #Q1 #critlib Currently trying to find out our ARL reporting requirements as there are implications for how we do accessions. STAY TUNED?!?!
  18. A1 We're ALA-accredited here at @iuils Not sure about the other types at IU, but probably standard ones for public Unis. #critlib
  19. Q1 main 1 is regional accreditation body but haven't been involved w that here, otherwise is program-by-prog (via subj liaisons) #critlib
  20. Q1: MPOW is a LIS program & everything revolves around ALA accreditation standards. Uni-level is Middle States, but no personal exp #critlib
  21. A1 Middle States is our biggie, plus others (ABET, etc.) for specific programs. As @ibeilin noted, library is often involved. #critlib
  22. @mauraweb Thanks - How could I forget Middle States! I was hoping you'd have the specifics :) #critlib
  23. As far as program review, admin at MPOW has always allowed us to create own IL outcomes. We build in #critlib to such outcomes.
  24. @lisalibrarian #critlib Q1: we have HLC accreditation. Each dept has an assessment plan with annual reporting since 1998.
  25. Oh yeah, GE assessment ongoing too, plus campus wide learning goals. #critlib
  26. Q2 addresses how those standards impact the library: How do accreditation, program review, governmental, etc. standards impact the library?
  27. Q2: How do accreditation, program review, governmental, etc. standards impact the library? #critlib Examples?
  28. Q2 - any impact outside of instruction? e.g. collections, staffing, space, etc. #critlib
  29. A2: with NEASC, focus on IL and assessment lends credibility to what we do in eyes of faculty and admin (unfortunately) #critlib
  30. @lisalibrarian ha well I do, but I like it because it's deserved, not because it is in a form they have to fill out :) #critlib
  31. Q2: If accrediting/review bodies don't *get* IL, it can really be a hindrance - reaching goals that aren't right for us #critlib
  32. @beccakatharine Right, and the Standards hanging around is going to make that job harder, esp if your program is pro-Framework. #critlib
  33. A2 at previous library HLC accreditation was a way to build relationships with faculty around issues of assess. and infolit #critlib
  34. A2 "we can help you assess learning/research skills" was our main pitch and we got on a document drafting committee #critlib
  35. A2: Our librarians have to work closely with departments/programs under review to insure that our collections fulfill reqs #critlib
  36. @lisalibrarian I'm not speaking first-hand, but currency and relevance seem paramount. Not sure what the benchmarks are. #critlib
  37. #critlib A2: By and large, my lib is left to our own devices. We devise SLOs, assessments, rubrics, benchmarks. We assess what we care abt.
  38. @kevinseeber Who decides what to assess & how? Is it mostly libr* or also other instruction faculty involved? #critlib
  39. @foureyedsoul All library driven. We partner w/ other programs, but they defer to us on info lit matters. #critlib
  40. Beyond accreditation we're in a similar spot to @kevinseeber in designing our own SLOs, assessment, etc. #critlib A2
  41. @cmcaffre @kevinseeber Doesn't that make it feel fairly insular, tho? Just affirming to librarian colleagues? #critlib
  42. @beccakatharine @kevinseeber yes and no. I don't think the SLOs are developed in a vacuum and are in a reaction to outside convos. #critlib
  43. @beccakatharine @kevinseeber Also we can participate in multi-institutional stuff (which we are) but I have doubts on how #critlib it is
  44. My experience same as @cmcaffre. Lib takes lead, but we talk a lot with other campus groups. @beccakatharine #critlib
  45. A2 pt2: revisiting GE outcomes allows us to make a case for #critlib rather than tradl info lit
  46. .@pumpedlibrarian @ginaschless A2 That is awesome, we're trying for this too but need to make more inroads. Slowly but surely. #critlib
  47. @mauraweb @ginaschless nice! are you guys on the gen ed commiittee for CUNY? can you vote? #critlib
  48. @pumpedlibrarian @ginaschless Yes, @718anne & I are on the college gen ed committee, uni gen ed is more...complicated. #critlib
  49. @ginaschless @mauraweb interesting... think tank, so not an official committee? I'm on our committee but as advisory membr, nonvote #critlib
  50. @trudysmoke oh wow you guys want to do all the assessing? I suppose you are smaller, I couldn't even imagine us assessing all GE #critlib
  51. A2 I see big diff betw prev job @ CC & current at ARL lib, where CC needs IL woven into accred but not as crucial @ current library #critlib
  52. @kshockey04 @lisalibrarian Its basically just a bar to get over to be part of the club. its not review #critlib
  53. @lisalibrarian @kshockey04 It would be considered bad form not to report. Its part of the membership deal--but it not accredidation #critlib
  54. @aasher @lisalibrarian I guess I was conceptualizing it as review - how good that process is obviously questionable. #critlib
  55. @lisalibrarian For us it seems that collections are most important for many of the program accreditations. #critlib
  56. @lisalibrarian @ibeilin same issue for us, program review focuses (maybe?) only on collections, not IL... need foundational texts + #critlib
  57. @pumpedlibrarian Yeah, my old job at ARL was all about books- didn't really look at info lit beyond # of classes. No SLO assessing. #critlib
  58. @pumpedlibrarian @lisalibrarian Many of our programs are tech-related and they want to be sure we don't have old stuff on shelves #critlib
  59. @ibeilin @lisalibrarian ha I haven't had to go through this yet b/c we switched to subj liaison model betw accred yrs for my progs #critlib
  60. @pumpedlibrarian @lisalibrarian This is one of the reasons why our library collections are kept lean and mean. not a research lib #critlib
  61. @ibeilin @pumpedlibrarian @lisalibrarian Interesting. I work w nrsing/sciences & don't weed 4 accred. Care more about db/ journals #critlib
  62. Weeding is good, but I walk thru the stacks and see the ebb/flow of interests of generations of librarians. That legacy has value #critlib.
  63. We're shifting toward IL and SLOs, lots of emphasis on measuring imact #critlib
  64. There will be increasing presure to demonstrate "value" #critlib
  65. @lisalibrarian good question! I've seen some arguments made based on results for additional staffing to support pilot initiatives #critlib
  66. @lisalibrarian at previous lib: program review, we were able to show "value" and use infolit success to argue for another position #critlib
  67. for GE assessment I had to argue w faculty as to why library should assess the IL competency... still not over that #critlib
  68. ++ but we want to know what they get and what they're struggling with and see if we can help #critlib
  69. Q2: as independent research lib, we don't have accreditation/instruction assessment, but have moved to document labor more fully--> #critlib
  70. We then move on to discuss how standards enable or are hindrances to critical instruction: Q3 - How do accreditation, program review, government standards, etc. impact critical library instruction? How are they hindrances, enablers, etc.?
  71. Q3: How do accred, prog rev, govt, etc. standards impact #critlib - how are they hindrances, enablers, etc.?
  72. Standards enable efficient reporting, frees up to do other things. So, structural assist to #critlib?
  73. .@edrabinski So Standards enable reporting of outcomes, thus freeing up time for #critlib learning... 1/2
  74. .@edrabinski ...is this a disconnect bw what's reported & what's actually happening? #critlib
  75. .@edrabinski Ah, ok. That makes sense. But your instruction generates non-#critlib assess data at same time #critlib = focus? #impressive
  76. @donnarosemary @edrabinski in same boat. we assess what we can, which is usually skills-y. #critlib stuff there just not measured same way
  77. a3: standards for assessment have oversight org's values embedded. those don't have #critlib in mind.
  78. @lisalibrarian (1/2) they historically have not. the biggest ex, ALA, has a specific set of values it institutionalizes. #critlib
  79. @lisalibrarian (2/2) #critlib is not what the CoA has in mind. It has the values as outlined by Lib bill of rights, etc.
  80. @kshockey04 got it. just pushing for whether is an "as practiced" or "just not possible" issue ... #critlib
  81. A3 The worry I experience w/ accred processes = somewhere in the reporting reqs, student learning in danger of being compromised. #critlib
  82. As with everything, how is power enacted within the structure? Whose intrests are reinforced #critlib
  83. A3: as @cmcaffre and I are finding, standards/outcomes need quant. data, (or aggregated) which #critlib often doesn't provide
  84. A3: Any may at times be incompatible with purpose of assessment.. #critlib .@beccakatharine
  85. @lisalibrarian @cmcaffre I wouldn't say irrelevant, but often contrary to the de-constructive nature of #critlib
  86. #critlib a3: IMO, most accred has little to do w/ learning and a lot to do with "counting what we can."
  87. A3 Seems standards could be hindrances & opportunities, depending on how used & ability to inflect with #critlib ideas #AlwaysContextual
  88. For us 2 diff things: assessment of student learning and evaluation of the library. The learning most imp. Collab evaded as support #critlib
  89. @bfister think this is key distinction library eval/QA vs student learning #critlib
  90. I'd say lso many things push in the other direction of #critlib - that's why we have #critlib
  91. @lisalibrarian @beccakatharine Can push it in a direction that impedes - but mostly it's more like two ships passing in the night #critlib
  92. @lisalibrarian sometimes they might align, but most often they do not because Am. libship is a classical liberal institution. #critlib
  93. @kshockey04 LIS programs get to set own outcomes tho w/i larger framework....? #critlib
  94. @kshockey04 intriguing ... i've seen a great deal more variation than it seems you have #critlib
  95. @lisalibrarian I see your point. Perhaps but not always a barrier but Incompatible=diff goals #critlib
  96. @lisalibrarian this is more of a rhetorical argument, but there is no american librarianship value that has #critlib at core
  97. @lisalibrarian there may be variance, but it is a niche. most people don't get it formally. #critlib
  98. Q3: #critlib Enabler: Required student engagement in assessment/eval/planning increases student agency.
  99. A3 #critlib Hinderance: Direct method SLOs confine students to predetermined readings, assignments, & outcomes.
  100. @LizLieutenant anything, any process or level of involvement that increases student agency is bound to be a good thing. #critlib
  101. @lisalibrarian @kshockey04 In our present moment I think many are antithetical to #critlib critlib as they reinforce and create inequality
  102. But, assessment when practiced well can be used to uncover inequalities, speaking truth to power etc #critlib
  103. @aasher Agreed! But could that ever be quant assessment? (I think not, but am curious). #critlib
  104. Put another way: assessment & accreditation are object lessons in why we need #critlib
  105. @lisalibrarian @beccakatharine assessment of learning can be #critlib but is often a diff question than program review type "value" assess.
  106. .@ginaschless @edrabinski Tho I cringe at this word in this context: is the non-#critlib assessment "authentic" then?(wtva that means:))
  107. .@edrabinski Do librarians do instruction session-level assessment in your program? #curious #critlib
  108. @kevinseeber I think we require mixed methods-- qual and quant can support each other #critlib
  109. @edrabinski @donnarosemary Curious about your exp as an LIS educator. Curriculum has some #critlib, yes? Does this influence assessment?
  110. @lisalibrarian I take specific issue with the ALA defintion of intellectual freedom and its incompatibility w/#critlib
  111. @lisalibrarian but accreditation as values-institutionalization arm in LIS ed favors intellectual freedom. #critlib
  112. @kshockey04 @lisalibrarian hmmm. that requires deep reflection. Does it mean #critlib has no place in the foundation culture?
  113. Hello #critlib from the west coast #c4l15 - there are many cross currents btw the two communities
  114. @rawdeal85 would love to see more direct collaboration between the two #critlib
  115. Question 4: What strategies do you have for articulating critical library practices within standards-based programmatic assessment?
  116. Q4: So, what strategies do you have for articulating #critlib practices within standards-based programmatic assessment? (if you do so)
  117. A4 If I can strike "standards-based" from the Q, 1 strategy is to articulate SLOs that are intentionally #critlib and measure those. 1/2
  118. A4 Ask questions re: assessment: who is calling for it? Who is it intended to benefit? Who will it actually benefit? #critlib
  119. A4 unsure but we're working on new SLOs that have more #critlib philosophy which will guide our assess.
  120. A4 For ex, we plan to embed privacy edu/advocacy into both IL SLOs and lib policies in near future; data=evidence we DONT keep data #critlib
  121. A4 In academic libs there's no escaping assessment, so it's worth the attempt to #critlib our assessments as much as we're able
  122. A4- Trying to rewrite rubrics to incorporate #critlib ideals. No more points for naming DB- now points for critiquing process of using DB.
  123. @lisalibrarian now there's a question! I love how you always tackle the tough stuff! :) #critlib
  124. A4 This semester that means joining group writing test q's based on the Standards & trying to carve out #critlib space there
  125. Just saying it's been done & is poss, & I even created #critlib-based rubric for campus, I think we worry more than we need 2 #critlib
  126. + even if #critlib assessments or rubrics aren't perfect, isn't the point of accreditation/large scale assess to show what we value?
  127. @lisalibrarian I thought Q was re: lib practices. We're Middle States here. We're able to define & measure our IL prog locally #critlib
  128. .@EamonTewell also important to consider how data we're collecting fits in the wider context of data, society, surveillence, etc. #critlib
  129. @nicholsonkp totally! and this goes back to the idea of needing to work more closely with faculty, escape the standard 1-shots #critlib
  130. @hsifnihplod I don't see #critlib as a monolith that needs broader validity - it is problematizing the broader validity we already have.
  131. @ibeilin @hsifnihplod @kshockey04 why not? Reflection & praxis can become operating procedure #critlib not necessarily subversive
  132. @kevinseeber @ibeilin @hsifnihplod @kshockey04 I imagine #critlib as a sort of ever-receding horizon. It's still based in librarianship, tho
  133. @kshockey04 Key question. If not, does #critlib really exist? Can it really accomplish goals that are good? Discussing this in #moocmooc
  134. @oksveta @EamonTewell esp.when it comes to who's collecting & what is being missed - #critlib can identify underrepresented groups
  135. @plutoHimself @hsifnihplod @kshockey04 until we radically reconfigure the society that creates american librarianship, #critlib must be subv
  136. A4: Standards obligate application of assessment results to plan/assess/develop/improve, so provide students opps to assess program #critlib
  137. I think classroom level assessment can be useful. Trick is doing it with student needs (not yours, not your institution's) in mind #critlib
  138. Something I'd love to try is asking students to design assessment tools and critique their validity #critlib
  139. Determining learning outcomes in advance seems in direct opposition to the notion of "student centered-ness" #critlib
  140. @EamonTewell having students create rubrics very valueable, again not sure how to do so with one-shots #critlib
  141. .@EamonTewell agree! and focus on formative assessment for this reason is most important here #critlib
  142. @ginaschless @EamonTewell maybe more important 2 have students begin thinking critically about services than actually create rubric #critlib
  143. To wrap up, our crit lib pitches!
  144. #critlib pitch--NYers, consider coming to @mnylc outreach & advocacy SIG tomorrow at 10am. I'm co-convening with @mappynyc.
  145. #critlib check out--if you all wanna proofread a paper on the rub between social justice and ALA intellectual freedom, pls contact.
  146. #critlib pitch: @kellymce & I co-editing a critical library pedagogy book! Submit proposals to us through March 31st!  http://critlib.tumblr.com/CFP 
  147. If u want more space to write your thoughts about the #critlib chat topic tonight (or any nite), we got u some space:  http://critlib.tumblr.com/submit