The Secret to My Survey Success

Surveys take a lot of critical thinking. You can’t just be in robot mode and “wing it” like it’s the Friday of Homecoming during your high school English class with students going wild. Just like any essay or presentation, you should make a draft or two before you create it in a survey program. Question design can’t just be about curiosity or it just sounding like a “good” question. They need to be questions that actually generate data that can tell you something. If questions are too general or unclear, then there are no real valid conclusions to be drawn. However, if a survey is repetitive and long, you might also lose the interest of your participants. Therefore, creating surveys takes critical thinking, but also discernment. Without much experience creating surveys, discernment is difficult on your own. So, it seems reasonable that when creating a survey,  you collaborate at some point in the process. Collaboration with a coworker or someone with just fresh eyes can help you be more specific, revise for clarity, and pinpoint missing questions.

I think I have it in my mind that as I progress as a professional that I need to be creative and clever all on my own. However, as I grow, I have learned to work smarter rather than harder. In order to actually be smarter, I need more than me. Lev Vygotsky speaks to me more and more the older I get. So, it both surprises me as well as does not surprise me that seeking social validation of questions for a survey is encouraged.  Why not ask my husband if my questions make sense? Why not view my work with a classmate for validation and revisions? Why not do a test run of my survey with fresh eyes? It not just makes sense; it also makes the task easier!

In fact, I think the hardest part of making a quality survey is working alone.

Advertisements

Is Quantitative Data Intuitive?

It’s hard to say that I’m not a “math person” when I love to 1) solve problems, 2) use logic, and 3) look at data. However, and this is a big HOWEVER, there something about creating data that does not feel intuitive. Learning most things, outside of musical instruments, typically feels intuitive for me. But, when reading a chapter on Analysis and Interpretation of Quantitative Data, my eyes start skimming for what seems important to highlight. The descriptions and advise given do not engage my attention or help me to envision what it would look like if I were actually doing the collection and displaying of data. I think in this instance, I need to do something more tangible, kinesthetic, with these new processes.

It is similar to following maps for me. I have spent many years traveling, mostly on my own, sometimes using hand held maps and sometimes using street maps at bus stations. Either way, I don’t really get lost. I see the map with my eyes, I study the main layout of the area, then I can envision the same thing in my head. It isn’t an exact copy, but I noticed enough to help me make good decisions on the way so that I don’t get turned around. However, when you put me inside a digital game and give me a map, I get lost. Seriously. I cannot figure out how to get to any location and often get separated from the group because I turned the wrong way for a split second. When I turned back, the group is gone. I will admit to running a zone for an hour without having actually done anything because I got stuck in some mountain region. I usually have to get my husband involved to make it make sense. It makes me feel totally incompetent, but I also know in real life that this is so easy for me.

I figure it will be the same with the data. Stuck in the textbook, I will just be lost but put me in a real life situation, I can handle it.

 

 

Using Surveys for Quality Data Collection

Before I read this chapter over surveys and questionnaires for research, I just thought surveys were annoying because the surveys I am thinking of are the ones from Subway where you can get a free cookie if you fill it out.  I totally get wanting feedback on how your store is doing, but honestly I don’t think it is a valid source for businesses to reward stores and employees. I used to work at places that gave incentives for surveys, but while you get the survey filled out, how real is the data? If I want a cookie, I’ll give you all 10s because I don’t care about to store’s improvement; I care about getting the free cookie. However, after reading this chapter, I think surveys and questionnaires could be pretty useful pending on the willingness of the participant to give real feedback. What incentives can we give to help get participants actually involved? I know if there is no incentive, I still don’t care about it! So, this chapter really got me thinking of ways to attempt to get genuine feedback from participants but also having a quality sampling. I don’t have answers yet; I think that comes with experience.

Quantitative Research Design

What seems most interesting about quantitative research is that it what it produces, if done ethically, are results that seem reliable and reproducible. So, if I am trying to solve a problem, and I use a fixed design to test out a solution, the design can (should) be replicated in other situations and environments. The process itself seems more controlled and intentional so that its results seems more credible and reliable. I think using this method will help to create generalizations that are general conclusions drawn from the evidence discovered in the research design. This kind of inferencing can help an overall understanding of whatever your topic/issue is. It can address what is typical but maybe it can’t address what is not typical. What I mean is that I often read or hear about practices that work in the classroom that I try out in mine, but I don’t get the same results. I don’t know if it matters that I teach low income students who speak two languages and often don’t value education. Were those conditions considered in developing those practical ways to teach students? More often than not, they were not considered. So I think quantitative research can help recognize patterns to help generalize what it is typical but I am not sure if that kind of research applies to the classroom at all times. So, I am drawn to fixed design for the structure and replication aspects of it, but I also recognize it is not an all-encompassing design model.

Ethics in Research

I think what I am most concerned about ethically when doing research is my own ignorance. My background in research is 100% historical research and/or literature analysis, no studies of humans. So, I haven’t been trained to be aware of possible effects of the research I will do. Similarly, I hadn’t considered that providing a technology intervention should be considered a form of treatment. If I was handing out pills, I would definitely be more cautious about testing out my theories on people, especially on students which is predominately my research field. I have done my NIH training, and it was a little overwhelming. In fact, my first response was “umm is this really the kind of research I want to do??” Ha! Then I remembered that I want to solve problems for teachers and students, so this kind of research is what it is going to take.

I have the same concerns about the IRB approval process. I am not interested in wasting the time of the faculty or the IRB committee by submitting an application that shows my ignorance. So, I really need to know my resources for constant review to help me in the IRB application as well as for my own research design. One of the requirements in our research is do not harm. It had never occurred to me that anyone in Learning Technologies could do harm! That is because my previous perspective of research had nothing to do with studying humans directly. Further, I am not clear on when to do IRBs. What is the timeline? What can you do when you are waiting on approval? During our meeting, Dr. Warren said when getting IRB approval on minors, it could take between 2-4 months. So, I am concerned about time management since my work will be focused on students.

Just Thinkin’ ’bout Research

At this stage in my career, I am definitely more concerned with practical research questions rather than theoretical. What draws me to practical research is my job. I currently work in the classroom. I am constantly evaluating and adapting my process even within the same day in a different class period. I have a great group of students who have a particular set of obstacles to overcome. So, when I think about spending my limited free time on research, I want to have practical answers to help my students. Further, I want to find real solutions that work in the classroom. I want to witness the effectiveness of some aspect I’ve researched for my specific students that actually works. What’s more, if it works for me, I want to share it. I don’t have a problem with theoretical research; it just feels less important than practical research.

 

I guess to start, my first research question is What is collaboration? I want to know what definitions are out there. I am specifically looking for versions that lean towards interdependent learning structures and assignment design. Then, I’d like to test out some version of interdependent learning style collaboration on a group of my current freshman in high school. I would like to take a 6 weeks, possibly the 6 weeks teaching Romeo and Juliet where I have two of my four classes integrating  collaborative assignments and two classes using the traditional group work style assignments. I’d like to do two for each side since time of day and class size really affects student participation and involvement. For example, first period of the day is one of the most difficult to get students to work with anyone. They are tired and don’t want to talk to anyone.  Also, the last period of the day is often more rowdy and harder to keep focused. Group work easily shifts to social time since they  just got back from lunch. So, I’d like a wider group for each style of teaching/learning. As far as data, I’d look at the level of analysis in their final project as well as the data from a multiple-choice test.

What to do, what to do?

The kind of research I am attracted to is research based on actual successes in the classroom. However, in class this week you talked about Education being a subset of Psychology. You described education as a way of influencing behavior and thinking, and therefore, theories of thinking should underpin our research projects. This is a new realm of research for me, so I think starting here is both efficient but also overwhelming. I have done zero amount of research in Psychology as it relates to Education. I wonder if this would be a better minor for my degree plan rather than business administration or educational leadership? I can’t know until I start reading.

The questions I am constantly asking are related to both read and writing and always collaboration. I have my own theory, based on experience as a student and a teacher, that learning is social. I can easily learn from a book. I can easily learn from doing. But, the most engaging way I have learned is by having to work with a group on an assignment that I could not truly do on my own. This type of learning was also connected to an immersive learning experience abroad. So, I wonder what kind of design creates those types of assignments? How do we design that? What theory is under all that? How do we create interdependent learning opportunities? How do we evaluate if it was an effective method? How do we measure growth with these types of assignments? How will these assignments support state testing in a high school setting? How do we draw in reluctant students to play their part without it damaging others in the process? Or can we? How do we really “grade” collaborative learning? Should we? Can we create immersive learning environments in a stationary classroom? What learning technologies help to create interdependent learning as well as immersive technologies?

I can see that there are two topics intermixing: collaboration and immersive learning. There is an overlap. When I get into gaming in the classroom as a method for collaboration that is where I start to delve into the immersive learning environments as well. That is when I come back to reading and writing as topics for research. I wonder how we could use immersive technologies to teach reading and writing separately but also maybe together!  I don’t know what is out there, so I need to start reading!

Real World Research

I am most interested in collaboration in the classroom. So far right now, collaboration is only a word that educators throw around but it has very little substance. Collaboration is more than group work, but what is it really? Further, how does technology support that, enable that, and hinder that? So, my research will be focused first on defining the term more concretely. Then, I’d like to see if there are different functioning versions of this definition. I’d like to then test out those theories in my own high school classroom. After those tests, I want to know how to more effectively train a variety of teachers in the real meaning and function of collaboration. This would include core, vocational, and elective teachers. Finally, I would like to support this research with suggest strategies for instructional design integrated with effective collaboration

As far as learning technologies, I am curious about available tools out there like VoiceThread or Quizlet Live that support collaborative learning. I am also curious about Computer Supported Collaborated Learning (CSCL). I don’t exactly know what that is, but Dr. Lin suggested I look into it for this topic. The common tool used today for “collaboration” is live documents. These kinds of tools are super cool, but I am wanting tools that create an interdependent learning environment rather than just a shared space.

What I found most enlightening about class and the first chapter of Real World Research is the emphasis on what our research should be about. I appreciate that this research is really focused on real-world problem solving. I do love research, but I feel that research for the sake of knowing is short-sighted. I can do that kind of work on my own time. But, research for the sake of problem solving is very motivating, and what’s more, it feels easier. You are telling I should be trying to solve problems, not just point them out. This gives more validity to why I chose this program over a strictly English PhD and that makes me feel confidant about my choice.

Control Yourself

Tonight I was watching the Gold Cup Quarter Final between the USA and El Salvador. It was not our best defensively, but it was one of the more aggressive games I’ve seen by the US Men’s National Team. If you watched, it was obvious why. El Salvador was clearly the little dog trying to aggravate the big dog. Literally, in the 57th minute, our Jozy Altidore was bitten on the shoulder while also having his nipple pinched by El Salvador’s Henry Romero. How in the world Jozy didn’t turn and punch this guy is beyond me. The ref didn’t see it, so there was no justice. Man! What is it that kept Jozy Altidore in line? Self-regulation. Like so many life skills, self-regulation takes time and practice before a person will find success. Altidore has had 12 years of practice, but my students probably have had none.

So what is it? Self-regulation is really just the ability to control yourself without needing some outside influence to guide you. No one had to hold Altidore back from Romero. He gritted his teeth and kept playing. In the real world, it means I can walk through a Target without having someone tell me NOT to steal something. I can decide for myself to pay for the things I want. In the classroom, it means students take the initiative for their own learning even if they’re not “feeling it.” When they start an assignment, they read the directions. When they get stuck, they ask questions. When they find success, they share it with their classmates. When they are trying to problem solve, the look to their classmates for support. In essence, when students self-regulate they are not dependent on a teacher for the answers.

Self-regulation can also influence others. When students are able to take ownership for their learning, it rubs off on the students around them. It is an odd affect since it is an outside influence regulating others, but since it is unintentional what is really going on is that other students are learning to regulate themselves by witnessing someone else do it. A teacher can help magnify this behavior by acknowledging it in the moment. She could even exaggerate the moment with a “Wow, so-n-so, you’ve really impressed me today! You got started so quickly!” or even drop a few tootsie rolls on a desk or two quietly acknowledging the behavior you want. Whatever it takes for students to observe and learn from the students already self-regulating is worth it because without the ability to self-regulate, students will never actually learn.

However, it cannot be expected that your students, whatever age they are, know how to self-regulate. It is a skill to be learned through constant coaching in the early stages and utilizing teachable moments when self-regulation was not practiced. For example, if I am teaching freshman in high school, I expect to have this same conversation every time I see a student:

Me: Hey, so-n-so, whatcha doin?

S-N-S: Oh… I’m working…

Me: Oh really? What specifically are you working on?

S-N-S: Umm, the assignment…

Me: Oh man, that’s great. Which part are you on?

S-N-S: Well, I just started, but I am confused.

Me: Oh really?? Man, what is confusing?

S-N-S: I’m not sure what I am supposed to be doing.

Me: Oh! Ok. Well what did the directions say? Oh, you didn’t read the directions? Ok. So, read the directions, I’ll be back in 2 minutes to check on you, and then I expect you to ask me specific questions about the directions so you can really get started.

Then what happens is I swing back by and by then the directions are read and the student has started. As the year goes on, this conversation happens again and again at different stages of the assignments. By the end of the year, most of my students have not only learned to self-regulate, but they have stopped asking me for help on topics they can Google search. I have a job as a teacher to be a guide. I cannot however be the motivation and regulation for a student when it comes to learning. Once they know how to regulate themselves, then students reach their goals often on their own. Without it, teachers and students alike will be stuck just dealing with discipline problems.

 

 

Part 1 Is Over

The main issue I had with project was the timing. Somehow I got a head and then I was behind. Then, I felt lost. Part of the struggle was that each document in the early stages contained so much duplicate information, I didn’t have a clear understanding of the role for each. The affect of this is that everything since has felt jumbled and over-complicated. I do most of this kind of thinking already when I plan my own class lessons. I don’t always go into such detail but I can see how dense the analysis task is now that I think it will make  my own classroom lessons more attune to learning gaps. I just wonder how much of it is necessary? By it, I mean all the different initial documents. Part of my hang up is that I designed a lesson that was for a subject and classroom environment that I am familiar with. Had I used a client that I was not a subject matter expert for nor familiar with their work space, I think all the documents might be more useful. I still question why each document requires duplicate information. Our textbook is called Rapid Instructional Design for a reason. It is meant to help us cut out what is extra and put into practice what is not. It makes me wonder if this course is actually created around this textbook. It might not be.

 

My experience with the first project has already influenced the second project in that I feel lost getting started. I am just feeling uncertain about what to do. It’s funny. If I ignore what I think I am “supposed” to be learning and just do what I normally do, then I discover a clear path in front of me. But, when I try to perform for others, I feel as though I am just standing on the stage alone having forgotten the song.

 

I think the strongest part of my first project is that it really pinpoints a specific skill for re-teaching. Having focused so narrowly will help the instructor evaluate whether this was the real issue or not. High school classrooms these days are more like science labs. You just gotta come up with a theory, design the experiment, then try it out. Maybe the outcome with be positive or maybe it won’t. Regardless, it will be a great start to slowly solving the problems in today’s classroom. However, I think the weakest aspect of the project was that it was so narrowly focused. What if this project totally misses the mark? What if I got so side tracked by what I thought rather than what the students needed, that I just waste their time? It is hard to know since school is not in session.