Context Description
Syracuse University Project Advance (SUPA) is an enhanced concurrent enrollment program linking Syracuse University (SU) with secondary school. Through this partnership, high schools can offer qualified seniors the opportunity to enroll in SU courses for university credit. Teachers who have qualified through SUPA as SU adjunct instructors teach enhanced concurrent enrollment university courses in high schools during the school day.
Preparing students to be college ready is one of the stated missions of Syracuse University Project Advance (SUPA). Working as a graduate research assistant in Research and Evaluation at Project Advance, I have been working (along with the team) on evaluating how effective we have been in meeting this mission. This has been done from the students and SUPA teacher’s perspectives. In the end, we hope to update/change the professional development sessions in the seminar in relation to college readiness skills.
We had conducted a quick program evaluation for this purpose with an online survey of Syracuse University freshmen that used to be SUPA students, and a focus group discussion with a small group of accounting teachers. Through this study we aimed to answer the question of whether SUPA courses provided students the skills necessary to be college ready. The overall results from this initial study showed that students did perceive their college readiness to have increased after taking SUPA courses.
This initial study gave us good insight into the issue, but left much more questions unanswered. Further, it was mainly limited to students in terms of representing their perspective. Now we have decided to expand upon this study in order to gauge how well the program is doing, in terms of meeting its mission of making the students college ready, from SUPA teacher’s perspective in detail.
In this new phase of the study, we expect to conduct focus group discussions with various groups of SUPA teachers- representing different subject areas. This will be followed up in a later phase with an online survey of the teachers. The main areas in which we are trying to gauge can be described mainly by the following questions:
- What is high school teacher’s current awareness of college readiness skills?
- What teaching methods are used for incorporating college readiness skills into their lesson?
- What is the teacher’s perception of their competence in incorporating college readiness skills into their lessons?
The results of this will give an insight into the understanding of teachers regarding college readiness and what they do to incorporate this into their lessons. This information in turn will provide a framework into the professional development needs of SUPA teachers, in terms of their college readiness skills.
The possible (and likely) instructional design outcomes could be incorporation of appropriate college readiness skills as instructions for professional development during the teacher seminar session, which are held every semester. Other possible instructional solutions could be the creation of webinars, or having online discussion groups, building a community of these teachers for participation on appropriate college readiness topic and so on.
Application of IDD&E
In one of the IDD&E courses, Techniques in Education Evaluation, I decided to take on an evaluation study in a study area of interest for Syracuse University Project Advance. This was a place that I worked as a graduate research assistant within the research and evaluation unit. I worked with my supervisor, Dr. Rob Seth Pusch, who himself is also an adjunct professor at the IDD&E department, to come up with a program evaluation. The resources available to me in terms of support made this endeavor more systematic and effective.
Thinking from ADDIE model perspective, this program evaluation was analyzed and designed with help from Dr. Pusch and we came up with a study area and a research design to get the information that we required. I had a small group of students from my class work with me and we did some literature review about college readiness. This helped us develop our study.
Our team from the Techniques in Education Evaluation course evaluation group quickly created a survey covering areas likes time management, task management, academic skills, college admission and financial management- all representing college readiness skills. We conducted the online survey with students and a focus group discussion with one group of teachers. Finally, we analyzed our data and presented our findings both to our class, and Project Advance. In doing all this, we passed through the analysis, design and development phases.
The Implementation was quick and we had out findings and recommendations based on the result. For the course and my project partners, this marked the end of this study. However, for me working in Project Advance, this was just a beginning. We were happy about how well this study was received, and in the enthusiastic participation of students. However, this was mostly a one sided study as we did not manage to get another perspective on this, from the teachers side, in great detail. I evaluated our study and its results, and have seen the shortcomings of the earlier evaluation. More importantly, we still do not have the types of information regarding college readiness skills for students that we need in order to be able to precisely develop professional development of teachers during seminar sessions.
We are now designing and developing the second study on this, focus group discussions with various groups of teachers regarding college readiness. This is an effort to be undertaken within our research and evaluation department in Project Advance. This will be followed up with an online survey of the teachers.
In the near future, these studies will all provide us information on the college readiness skill requirements of the teachers and help us develop a (instructional) solution which will likely take the form of professional development lesson during seminar sessions. If this turns out to be the solution, this will not require significant resources, as it will be built into the existing session of college readiness that already exists as part of the seminar. The content development should not be a major hurdle as we have experts on this area facilitating the seminars already. The study might show new college readiness skills that need to be included, while it may also show ones that need to be dropped form the existing structure. The time for this college readiness piece should easily be incorporated into the day program without much hassle even if it has to be longer than before.
Knowledge Gains
The college-readiness related projects have helped me work on improving mainly 6 instructional designer competencies. There are 22 competencies set by the (ibstpi) Instructional Designer Standards Competencies guide (http://ibstpi.org). Various activities and projects through the program did manage to touch on most of the 22 competencies at various stages of the MS program. Some of the competencies were well known coming in, and others got developed as we progresses through the program.
The college readiness studies have helped Syracuse University Project Advance build a better understanding of how college readiness skills have actually been developed from SUPA courses. The SUPA teacher seminars held every semester at Syracuse University helps teachers’ participate in professional development activities, which includes on college readiness. The studies have used data collection and analysis as a part of the research into college readiness. The competency “ Update and improve knowledge, skills, and attitudes pertaining to the instructional design process and related fields” was used in these studies and contributed to knowledge gain. They will further likely be used to create instructional solutions.
The college readiness studies used the scientific research techniques that involved using data collection using various methods from different stakeholders with the aim of getting the required information as set by our study objective. This was followed up by analysis of the data from our study. The competency “ Apply data collection and analysis skills in instructional design projects” was explored quite well. In the end of the college readiness study, we should have the data that will allow us to analyze the proper instructional requirement for the appropriate college readiness skill that teachers would require.
The college readiness studies were conducted as a research study that meant they had to cover different perspectives, in our case, mainly the students and teachers. The problem was defined in terms of what needed to be done by these main stakeholders in order to achieve the stated mission of Project Advance of college readiness. Analysis of the data collected followed up by making appropriate recommendations and reporting the findings are always integral parts of these studies. The competency “ Conduct a needs assessment in order to recommend appropriate design solutions and strategies” was covered well by these studies. Further, a major focus of the study was to figure out the required college readiness skills that teachers would require to learn as a part of professional development during the semester seminar.
The college readiness studies are assessed and evaluated at various stages of the study. For the first study, discussions were held with experts, group members and others, and a program evaluation rubric was used as a standard. In the end, the whole study was evaluated depending upon how the whole research went and the results obtained. This was used to design a new study with the learning’s from the earlier study in mind. The competency “Evaluate instructional and non-instructional interventions” played an important role in the studies. Going forward, this second phase of the study will be evaluated and after implementation (likely in the form of college readiness skills professional development of teachers during the seminars), evaluation will need to be done not only of the teachers learning, but how well they are able to transition that into their instructions to be taught to the students.
The college readiness studies are assessed and evaluated, and learning is used for future studies. The competency “ Revise instructional and non-instructional solutions based on data” is a fundamental part of the college readiness studies process. The studies will likely lead to the revision of the teacher seminar’s to include the required college readiness topics. It can also lead to other solutions such as creation of webinars, online discussion groups, and other ways of building participation and sharing among the teachers on this topic.
The college readiness studies aims to align our practice with the stated mission objective of Syracuse University Project Advance. The main thing we are looking to do is to provide professional development related to college readiness in areas that are lacking. The competency “ Implement, disseminate, and diffuse instructional and non-instructional interventions” is an integral part of the whole process. After the studies, one of the (likely) implementation will take place as instructional college readiness skills development lessons during the teacher seminars.
Besides these competencies, many other competencies are touched upon by these studies to some extent. The knowledge gain in this direction is occurring in a significant manner.
Final Reflection
Coming from a research and evaluation background, most of the subjects covered by the Master of Science in Instructional Design, Development and Evaluation seemed unfamiliar to me at first. The learning’s from the first semester with the core courses did touch upon an array of new things. But as we moved to implementation and the program progressed, the skills and knowledge’s provided by them simply gave better ways of doing things that I had already done in the past, and no longer felt so unfamiliar.
The college readiness studies were chosen since they were directly used in the IDD&E courses, but also elaborated in my work at Project Advance. However, the learning’s from the program have come from many other projects besides these as well. The general ADDIE model, along with other specific models, really impressed me with their applicability in different scenarios. The ibstpi competencies is a list that will definitely come in handy going forward as it is a complete list of possible areas which I would need to be aware of, and develop, in the future.
Some areas of the ibstpi competencies are more familiar to me at this point, while others still require a lot of work. I would say I have a strong understanding in the areas covered by the competencies in professional foundations, planning & analysis and evaluation & implementation. The areas that I believe I need to continue to work on and improve are in design & development and management.
As I look to move away from the academic world and back into professional work life, I feel confident that I can use the skills, competencies and models in an array of areas that are so frequently used in work-life. I have definitely built a strong base from where I can climb higher. The MS has also opened a whole new area of potential fields and roles that I could get into. The knowledge, skills and tools that I have in my disposal now are a vital asset for me going forward.