Using Surveys for Quality Data Collection

Before I read this chapter over surveys and questionnaires for research, I just thought surveys were annoying because the surveys I am thinking of are the ones from Subway where you can get a free cookie if you fill it out.  I totally get wanting feedback on how your store is doing, but honestly I don’t think it is a valid source for businesses to reward stores and employees. I used to work at places that gave incentives for surveys, but while you get the survey filled out, how real is the data? If I want a cookie, I’ll give you all 10s because I don’t care about to store’s improvement; I care about getting the free cookie. However, after reading this chapter, I think surveys and questionnaires could be pretty useful pending on the willingness of the participant to give real feedback. What incentives can we give to help get participants actually involved? I know if there is no incentive, I still don’t care about it! So, this chapter really got me thinking of ways to attempt to get genuine feedback from participants but also having a quality sampling. I don’t have answers yet; I think that comes with experience.

Advertisements

Quantitative Research Design

What seems most interesting about quantitative research is that it what it produces, if done ethically, are results that seem reliable and reproducible. So, if I am trying to solve a problem, and I use a fixed design to test out a solution, the design can (should) be replicated in other situations and environments. The process itself seems more controlled and intentional so that its results seems more credible and reliable. I think using this method will help to create generalizations that are general conclusions drawn from the evidence discovered in the research design. This kind of inferencing can help an overall understanding of whatever your topic/issue is. It can address what is typical but maybe it can’t address what is not typical. What I mean is that I often read or hear about practices that work in the classroom that I try out in mine, but I don’t get the same results. I don’t know if it matters that I teach low income students who speak two languages and often don’t value education. Were those conditions considered in developing those practical ways to teach students? More often than not, they were not considered. So I think quantitative research can help recognize patterns to help generalize what it is typical but I am not sure if that kind of research applies to the classroom at all times. So, I am drawn to fixed design for the structure and replication aspects of it, but I also recognize it is not an all-encompassing design model.

Ethics in Research

I think what I am most concerned about ethically when doing research is my own ignorance. My background in research is 100% historical research and/or literature analysis, no studies of humans. So, I haven’t been trained to be aware of possible effects of the research I will do. Similarly, I hadn’t considered that providing a technology intervention should be considered a form of treatment. If I was handing out pills, I would definitely be more cautious about testing out my theories on people, especially on students which is predominately my research field. I have done my NIH training, and it was a little overwhelming. In fact, my first response was “umm is this really the kind of research I want to do??” Ha! Then I remembered that I want to solve problems for teachers and students, so this kind of research is what it is going to take.

I have the same concerns about the IRB approval process. I am not interested in wasting the time of the faculty or the IRB committee by submitting an application that shows my ignorance. So, I really need to know my resources for constant review to help me in the IRB application as well as for my own research design. One of the requirements in our research is do not harm. It had never occurred to me that anyone in Learning Technologies could do harm! That is because my previous perspective of research had nothing to do with studying humans directly. Further, I am not clear on when to do IRBs. What is the timeline? What can you do when you are waiting on approval? During our meeting, Dr. Warren said when getting IRB approval on minors, it could take between 2-4 months. So, I am concerned about time management since my work will be focused on students.

Just Thinkin’ ’bout Research

At this stage in my career, I am definitely more concerned with practical research questions rather than theoretical. What draws me to practical research is my job. I currently work in the classroom. I am constantly evaluating and adapting my process even within the same day in a different class period. I have a great group of students who have a particular set of obstacles to overcome. So, when I think about spending my limited free time on research, I want to have practical answers to help my students. Further, I want to find real solutions that work in the classroom. I want to witness the effectiveness of some aspect I’ve researched for my specific students that actually works. What’s more, if it works for me, I want to share it. I don’t have a problem with theoretical research; it just feels less important than practical research.

 

I guess to start, my first research question is What is collaboration? I want to know what definitions are out there. I am specifically looking for versions that lean towards interdependent learning structures and assignment design. Then, I’d like to test out some version of interdependent learning style collaboration on a group of my current freshman in high school. I would like to take a 6 weeks, possibly the 6 weeks teaching Romeo and Juliet where I have two of my four classes integrating  collaborative assignments and two classes using the traditional group work style assignments. I’d like to do two for each side since time of day and class size really affects student participation and involvement. For example, first period of the day is one of the most difficult to get students to work with anyone. They are tired and don’t want to talk to anyone.  Also, the last period of the day is often more rowdy and harder to keep focused. Group work easily shifts to social time since they  just got back from lunch. So, I’d like a wider group for each style of teaching/learning. As far as data, I’d look at the level of analysis in their final project as well as the data from a multiple-choice test.

What to do, what to do?

The kind of research I am attracted to is research based on actual successes in the classroom. However, in class this week you talked about Education being a subset of Psychology. You described education as a way of influencing behavior and thinking, and therefore, theories of thinking should underpin our research projects. This is a new realm of research for me, so I think starting here is both efficient but also overwhelming. I have done zero amount of research in Psychology as it relates to Education. I wonder if this would be a better minor for my degree plan rather than business administration or educational leadership? I can’t know until I start reading.

The questions I am constantly asking are related to both read and writing and always collaboration. I have my own theory, based on experience as a student and a teacher, that learning is social. I can easily learn from a book. I can easily learn from doing. But, the most engaging way I have learned is by having to work with a group on an assignment that I could not truly do on my own. This type of learning was also connected to an immersive learning experience abroad. So, I wonder what kind of design creates those types of assignments? How do we design that? What theory is under all that? How do we create interdependent learning opportunities? How do we evaluate if it was an effective method? How do we measure growth with these types of assignments? How will these assignments support state testing in a high school setting? How do we draw in reluctant students to play their part without it damaging others in the process? Or can we? How do we really “grade” collaborative learning? Should we? Can we create immersive learning environments in a stationary classroom? What learning technologies help to create interdependent learning as well as immersive technologies?

I can see that there are two topics intermixing: collaboration and immersive learning. There is an overlap. When I get into gaming in the classroom as a method for collaboration that is where I start to delve into the immersive learning environments as well. That is when I come back to reading and writing as topics for research. I wonder how we could use immersive technologies to teach reading and writing separately but also maybe together!  I don’t know what is out there, so I need to start reading!

Real World Research

I am most interested in collaboration in the classroom. So far right now, collaboration is only a word that educators throw around but it has very little substance. Collaboration is more than group work, but what is it really? Further, how does technology support that, enable that, and hinder that? So, my research will be focused first on defining the term more concretely. Then, I’d like to see if there are different functioning versions of this definition. I’d like to then test out those theories in my own high school classroom. After those tests, I want to know how to more effectively train a variety of teachers in the real meaning and function of collaboration. This would include core, vocational, and elective teachers. Finally, I would like to support this research with suggest strategies for instructional design integrated with effective collaboration

As far as learning technologies, I am curious about available tools out there like VoiceThread or Quizlet Live that support collaborative learning. I am also curious about Computer Supported Collaborated Learning (CSCL). I don’t exactly know what that is, but Dr. Lin suggested I look into it for this topic. The common tool used today for “collaboration” is live documents. These kinds of tools are super cool, but I am wanting tools that create an interdependent learning environment rather than just a shared space.

What I found most enlightening about class and the first chapter of Real World Research is the emphasis on what our research should be about. I appreciate that this research is really focused on real-world problem solving. I do love research, but I feel that research for the sake of knowing is short-sighted. I can do that kind of work on my own time. But, research for the sake of problem solving is very motivating, and what’s more, it feels easier. You are telling I should be trying to solve problems, not just point them out. This gives more validity to why I chose this program over a strictly English PhD and that makes me feel confidant about my choice.

Control Yourself

Tonight I was watching the Gold Cup Quarter Final between the USA and El Salvador. It was not our best defensively, but it was one of the more aggressive games I’ve seen by the US Men’s National Team. If you watched, it was obvious why. El Salvador was clearly the little dog trying to aggravate the big dog. Literally, in the 57th minute, our Jozy Altidore was bitten on the shoulder while also having his nipple pinched by El Salvador’s Henry Romero. How in the world Jozy didn’t turn and punch this guy is beyond me. The ref didn’t see it, so there was no justice. Man! What is it that kept Jozy Altidore in line? Self-regulation. Like so many life skills, self-regulation takes time and practice before a person will find success. Altidore has had 12 years of practice, but my students probably have had none.

So what is it? Self-regulation is really just the ability to control yourself without needing some outside influence to guide you. No one had to hold Altidore back from Romero. He gritted his teeth and kept playing. In the real world, it means I can walk through a Target without having someone tell me NOT to steal something. I can decide for myself to pay for the things I want. In the classroom, it means students take the initiative for their own learning even if they’re not “feeling it.” When they start an assignment, they read the directions. When they get stuck, they ask questions. When they find success, they share it with their classmates. When they are trying to problem solve, the look to their classmates for support. In essence, when students self-regulate they are not dependent on a teacher for the answers.

Self-regulation can also influence others. When students are able to take ownership for their learning, it rubs off on the students around them. It is an odd affect since it is an outside influence regulating others, but since it is unintentional what is really going on is that other students are learning to regulate themselves by witnessing someone else do it. A teacher can help magnify this behavior by acknowledging it in the moment. She could even exaggerate the moment with a “Wow, so-n-so, you’ve really impressed me today! You got started so quickly!” or even drop a few tootsie rolls on a desk or two quietly acknowledging the behavior you want. Whatever it takes for students to observe and learn from the students already self-regulating is worth it because without the ability to self-regulate, students will never actually learn.

However, it cannot be expected that your students, whatever age they are, know how to self-regulate. It is a skill to be learned through constant coaching in the early stages and utilizing teachable moments when self-regulation was not practiced. For example, if I am teaching freshman in high school, I expect to have this same conversation every time I see a student:

Me: Hey, so-n-so, whatcha doin?

S-N-S: Oh… I’m working…

Me: Oh really? What specifically are you working on?

S-N-S: Umm, the assignment…

Me: Oh man, that’s great. Which part are you on?

S-N-S: Well, I just started, but I am confused.

Me: Oh really?? Man, what is confusing?

S-N-S: I’m not sure what I am supposed to be doing.

Me: Oh! Ok. Well what did the directions say? Oh, you didn’t read the directions? Ok. So, read the directions, I’ll be back in 2 minutes to check on you, and then I expect you to ask me specific questions about the directions so you can really get started.

Then what happens is I swing back by and by then the directions are read and the student has started. As the year goes on, this conversation happens again and again at different stages of the assignments. By the end of the year, most of my students have not only learned to self-regulate, but they have stopped asking me for help on topics they can Google search. I have a job as a teacher to be a guide. I cannot however be the motivation and regulation for a student when it comes to learning. Once they know how to regulate themselves, then students reach their goals often on their own. Without it, teachers and students alike will be stuck just dealing with discipline problems.

 

 

Part 1 Is Over

The main issue I had with project was the timing. Somehow I got a head and then I was behind. Then, I felt lost. Part of the struggle was that each document in the early stages contained so much duplicate information, I didn’t have a clear understanding of the role for each. The affect of this is that everything since has felt jumbled and over-complicated. I do most of this kind of thinking already when I plan my own class lessons. I don’t always go into such detail but I can see how dense the analysis task is now that I think it will make  my own classroom lessons more attune to learning gaps. I just wonder how much of it is necessary? By it, I mean all the different initial documents. Part of my hang up is that I designed a lesson that was for a subject and classroom environment that I am familiar with. Had I used a client that I was not a subject matter expert for nor familiar with their work space, I think all the documents might be more useful. I still question why each document requires duplicate information. Our textbook is called Rapid Instructional Design for a reason. It is meant to help us cut out what is extra and put into practice what is not. It makes me wonder if this course is actually created around this textbook. It might not be.

 

My experience with the first project has already influenced the second project in that I feel lost getting started. I am just feeling uncertain about what to do. It’s funny. If I ignore what I think I am “supposed” to be learning and just do what I normally do, then I discover a clear path in front of me. But, when I try to perform for others, I feel as though I am just standing on the stage alone having forgotten the song.

 

I think the strongest part of my first project is that it really pinpoints a specific skill for re-teaching. Having focused so narrowly will help the instructor evaluate whether this was the real issue or not. High school classrooms these days are more like science labs. You just gotta come up with a theory, design the experiment, then try it out. Maybe the outcome with be positive or maybe it won’t. Regardless, it will be a great start to slowly solving the problems in today’s classroom. However, I think the weakest aspect of the project was that it was so narrowly focused. What if this project totally misses the mark? What if I got so side tracked by what I thought rather than what the students needed, that I just waste their time? It is hard to know since school is not in session.

Second Life Reflection – Use the Tool or Be a Tool

We had a class meeting in Second Life. I was super stoked about it just because it was something different. I was super stoked because it was Virtual Reality. I love to play MMORPGs (massively multiplayer online role playing games) so having class in Second Life would be/could be fun. As entertainment, Second Life was interesting because you can explore for the sake of exploring. You can walk, run, and even fly. You can pick your own character and look however you want to look. The virtual world is virtually wide open. But, as a teaching tool? From what I explored, it’s not great. Overall, I would say this is just another tool that must be utilized intentionally, not used to replace a teacher which is often how teacher view technology.

The first issue I had in the first zone we explored was the lack of purpose. Our immediate purpose was to just explore, to see what we would see and consider how it was being used instructionally. Instructionally, this wasn’t really being used. There didn’t seem to be any game plan for how to use the space we were in. Namely, there were no objectives. In a game like World of Warcraft (WOW), the game play is generally open. You can go in whatever direction you chose and you can pursue whatever task you’d like. But, at least in W.O.W. you actually are presented with tasks. Random characters give you quests in different zones. You can choose which ones to pursue and which ones to wait on. There are also side quests that ask you to hunt or forage so that you can just roam different zones with a purpose. In the Pavilion we were in at the beginning of our exploration, it wasn’t really clear what we should or could do. There was no real guidance, no goals, and no objectives. So, as far as instructional design, it wasn’t clear, so ultimately the design was a failure.

Something else I noticed in the first zone was there were items that were clickable. That is as much interaction as was available. Point and click is not terrible, but is a point and click environment worth altering a classroom environment? Student engagement will last as long as the clickable items are interesting, but if what opens is just text to read, students will just check out. If we are willing to go through all the trouble of shifting to a virtual reality, it must be so that we can do something now that we couldn’t without the VR environment. This point and click environment is just a substitution for clicking open a link inside a Learning Management System (LMS) like Canvas or Blackboard. So, what’s the point?

The last zone we traveled to had a little more design to it. There was clearly a campus with different buildings that had different purposes. There were more signs and guides along the way. One particular building was like a gallery. You could walk through following the big red arrows and click on the different displays. It would tell you about each display. This is a great start for using the tool rather than expecting the tool to be the teacher. Someone clearly planned this space with a purpose and an audience in mind. They made clear choices about how to share the information they wanted to share. This space was an example of where so many teachers are these days: able to use the tool to do something we couldn’t have done before. However, what this space was lacking and most teachers are still lacking, is the challenge of what to do with the information being shared. Technology integration is not just about streamlining assignments and data, and it is not just about trying to engage students on a different level. There has to be something more. We must have our students apply and create something with this new knowledge. If we want to redefine education, we cannot just redesign how we share information. We must also redesign what we have students do with the information.

Overall I think spaces like Second Life have so much potential, but it is the teacher or instructional designer that must intentionally use the space for their own objectives, not expect the tool to take over for the teacher. During our meeting, Dr. Warren talked about needing a narrative in order for these kinds of environments to work. It is the narrative that provides the parameters, the purpose, the goals, and the context of the game play. Without it, we are virtually stuck in the wilderness like contestants in Alone. We have some tools, and we want to survive, but without a purpose, we will not make it to the end.

Method of Loci

I am comfortable with visualization. I think it comes from being a team-sport athlete. Somewhere along the way a coach of mine showed us how to visualize the game. I learned to see the space on the field and imagine what to do with it before I did it. I learned to see the game in front of me and create the movement. I was in control. It was up to me to be the instigator of plays and mover of the ball.  However, this kind of visualization is different that the Method of Loci used to remember Wilson’s Situated Instructional Design. The Method is meant to help you memorize ideas by assigning them to images, to movement, in order to create almost like a memory. This kind of visualization creates the past as if it really happened. You take something you are so familiar with that the memory of it is like a photograph in your hand. Then you superimpose new ideas into the memory in order to recapture something important. Hopefully, you can then refer back to the created memory because it now has emotional meaning to it that it didn’t before.

This is a fascinating concept, but then I think about what my students would do with it. I was that student that thought things like this were dumb. I wonder what my students would do in class if I walked them through this kind of memory work? Since meeting the Internet, my students can’t remember my name even by the end of the year. Would they scoff at this kind of exercise? Or would they take the time to create something with it… like new knowledge?

This method could work for them, but for me I hadn’t been specific enough with my time and place. I don’t have a favorite room. I don’t have a lot of favorites anyway. I do however have locations that stick to me like peanut butter on a dog’s tongue. But, these emotional palaces are too broad for this exercise. I can imagine myself as student staring back at me the teacher refusing to visualize. When prompted, my student self would say, “I don’t have a favorite room.” My teacher self would just say, “Use any place that you can clearly remember…” Then my student self would find some new response that is just as dense and unimaginative as the last excuse. And she may pretend to do the exercise, but that doesn’t really count does it?

 

So does this work? Could it be used? Possibly, but I’d have to do a lot of my own pre-game visualization to prepare for my most stubborn students.