Hearne ISD-Elementary- Marlise Golden

1.       Describe the purposes for and various stages of formative evaluation of technology plan.

Formative evaluation is also useful in analyzing learning materials, student learning and achievements, and teacher effectiveness. Formative evaluation is primarily a building process which accumulates a series of components of new materials, skills, and problems into an ultimate meaningful whole. - Wally Guyot (1978)A formative evaluation aims to improve the instruction, whereas summative evaluation aims to prove the worth of the instruction (Rieber, n.d.).

Since the technology plan is currently under development for Hearne ISD which will include the elementary school’s technology goals, three cycles of data collection will be used to analyze to make appropriate revisions to the new plan with a projection date for the upcoming school year:

·         One-on-One Evaluation

·         Small group evaluation

·         Field test/trial

Questionnaires, surveys and interviews will be conducted to ensure the data collected is accurate and valid.  There should be at least three formative evaluation stages used for Hearne ISD and the elementary school technology plan that has a projection date of fall 2012 for the 2012-2013 school years. Since the Technology Plan is not currently developed interviews and surveys have been conducted to find the needs.

  One-on-One Evaluation of hiring and maintaining a highly qualified Instructional 

Technologist/computer lab teacher conducted by principal. 

After hiring for the position of an Instructional Technologist, the principal will conduct evaluations of the instructional technologist by observing the IT assisting teachers with implementation of more technology within their lessons and or facilitation of training staff on the usage of new/old technology devices and or software.

Computer lab teacher- evaluations conducted by principal, mentor and master teachers according to the guidelines and rubrics of the TAP program that is in place for the entire district. The evaluators will be determining if the TEKS are being correctly taught in alignment with the current curriculum, CScope. 

 Small Group Evaluations conducted by a group to include Master teachers (subject-matter experts)

The Master Teachers will conduct small group evaluations on the Instructional Technologist and Computer lab teacher after the one-on-evaluations have been completed.  This will be to determine if the findings from the pre-hire surveys were correct to determine the need for both positions.  There will be interviews conducted for each grade level (1 teacher per subject area)

Target audience questions for the personal interviews:

* All surveys have to be completed after the new technology plan has been adopted and in place which will hire someone for both positions listed below.

Instructional Technologist

1. Have you had the opportunity to utilize the services of the Instructional Technologist? If yes please explain the assistance provided for you?

2. (if yes) Did the information or help you received from the technologist assist your needs in your classroom?

3. (if no to question 1) Are you planning any lessons in the future that will need the assistance of the Instructional Technologist?

4. Has the Instructional Technologist made themselves easily attainable to you and other staff members and if so by what means?

5. Do you feel more or less comfortable knowing that there is an Instructional Technologist available to assist teachers with their lessons?

Computer Lab Teacher

1. Have the student’s computer skills increased since the hiring of a new teacher for the computer lab.

2. Do the students discuss whether they are learning new skills in computer lab?

3. Have students learned their log in/password information?

4. Are the students using more computer terminology when using the computers in your classroom?

2.       Describe your instruments used in a formative evaluation. (see question 1)

3.       Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.



Since Hearne ISD currently does not have a Technology or Professional Development Plan in place and one of the projected goals that have been written is to hire an Instructional Technologist, after the position is filled a professional survey should be conducted to determine if this position has a positive effect on each campus especially the elementary campus where the majority of the students are in attendance. This feedback will come from the teachers after the first semester to give adequate amount of time for the Instructional technologist to assist as many teachers as possible.  This will allow more feedback from a variety of teachers on campus. The Master Teachers (3) should also be required to take the survey to determine if the new position has helped them in any issues they may have with lessons for TAP clusters.

Teacher Survey (professional instructors)
*Answers can only be provided after the position of Instructional Technologist has been created and filled.


1. Did the Instructional Technologist give detailed instructions during the Mini-Teach?
2. Did the Instructional Technologist provide handouts and or examples for the lesson being taught?
3. Did the Instructional Technologist suggest other teacher resources to use to assist your learning in the Mini-Teach to use in the classroom to implement more technology in your lessons?
4. Was the Mini-Teach requested by you or was it made mandatory for professional development for your grade level, subject area, or campus?
5. Did you find that this method of professional development was effective enough for immediate implementation in the classroom?
6. Is the Instructional Technologist effectively helping teachers meet the goals of the technology plan to create lifelong learners in a digital world?

After the surveys are compiled and evaluated, the results will determine if the position was actually an asset to the technology plan which will determine if the position will continue to be a part of the district or dissolved for the next school year or at the end of the currently adopted Technology Plan.


Resources

Guyot, W. (1978).

Rieber, L. (n.d.). Designing and Conducting Formative Evaluations. Retrieved October
29, 2011, from
http://www.lrieber.myweb.uga.edu/edit6170/ppt/formative_evaluation.ppt

Elementary - Donzella Manuel Week 11

Revise Technology Plan /  Design and Conduct Summative Evaluation

1. Describe the purposes for and various stages of formative evaluation of technology plan.

The purposes for a formative evaluation of technology plan is to allow you to gather feedback on whether you are meeting your goals and objectives during the planning and development of your technology plan rather than waiting until the technology plan is complete. Formative evaluations assess what works and what does not work for your technology plan.

There are four stages of formative evaluation:

Stage 1 -  Expert review(s) / Internal review (Technology Plan Committee)
The Technology Plan Committee review the technology plan for accuracy and completeness.

Stage 2 -  One-to-One evaluation
The designer works with individual learners to obtain data to revise the technology plan. The learner at a time reviews the content of the lessons using technology and give comments upon it. The purpose is to identify problems in the instruction, such as typographical errors, unclear sentences, poor or missing directions, etc.

Stage 3 - Small group evaluation / Tutorial and small-group tryouts
During small group evaluation of 8-20 learners/teachers (target population), the evaluator tries out the instruction with the group of learners and records their performances and comments. The output of a small group evaluation is a revised instructional lesson based upon time, performance and attitude.

Stage 4 - Field trial / Field test / Operational tryout
During a field test instruction is evaluated in the same environments in which it will be used when it is finished. The goal of the field trial is effective instruction the yields desired levels of learner achievement and attitudes and that functions as intended in the learning setting.
          

2. Describe your instruments used in a formative evaluation.

Formative evaluation relies on many of the same techniques that other forms of assessment use. Evaluation instruments that are used for a school/district technology plan is surveys, interviews, monitoring/observation, reports, logs. The instruments I used in the formative evaluation for my technology plan is survey/questionnaire, interviews, log and monitoring/observation. 

3. Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.

 The accountability methods and measures that will be used to evaluate the extent to which activities are effective are:

Technology
Integrating technology into curriculum and instructions, I used a survey/questionnaire, interviews monitoring/observation, and a log to collect data during the formative evaluation stage.

Funding
A carefully developed technology plan will help to make sure that my district have secured the resources needed. The budget must include estimated costs with available and potential funding sources for all areas addressed in the plan. I used monitoring/observation of the elementary schools technology budget during the formative evaluation stage.

Management
Increasing the ability of teachers to teach and learn technology with professional development, I used a survey, monitoring/observation and a log to collect data during the formative evaluation stage.


References:

Dick, W. and Carey, L. (1996). The Systematic design of instruction, 4th ed. New York: Harper Collins Publishing.

TPESC. (n.d.). Technology Plan Components - Evaluation. Retrieved March 3, 2012, from http://tpesc.esc12.net/eplan/Evaluation.html

Katy ISD- Ashley Peabody

1. Describe the purposes for and various stages of formative evaluation of technology plan.


From my educational experiences I was taught the the purpose of formal evaluation is not to assign grades but to identify the different weaknesses in order to make improvements. According to Dick and Carey (2001) said that the main purpose of formative evaluation is to find the weaknesses in the technology plan in order to make improvements. There are six main stages of formative evaluation:design review, expert review, one-to one, small group, field trails, and ongoing evaluation. The first stage is design review. This stage is where it is critical to answer the following questions; "Does the instructional goal match the problem identified in the needs assessment? Does the learner & environmental analysis match the audience? Does the task analysis include all the prerequisite skills? Are the test items reliable and valid, and do they match the objectives?" These questions are important because it helps evaluate the technology plan to make sure it is working effectively. The purpose of the small group stage is to look for the "effects be changes made in the one-to-one review and to identify any remaining learning problems". At the stage of field trails the administrators need to look for the changes and effects made in small groups and can the it be used the way it was attended. Finally the most critical stage is ongoing evaluation. This is evaluated be the :project size, life span of content, audiences change, one-to-one, small group tryouts, and field trails" (Dick and Carey, 2001). These evaluations are critical in order to make the technology plan a success!

2. Describe your instruments used in a formative evaluation.

There are a variety of different tools that can be utilized for formative evaluation. Tools include: questionnaires, interviews, focus groups observations, walk-through, rubrics, and reviews. There is not a series of specific tools that can be used when formal evaluating the technology plan. It should be based off the needs of the plan to determine how it will be evaluated (Evaluation Toolbox, 2010). The tools I plan to utilize while formally evaluating my technology plan is to interview, use a rubric, and to have a questionnaire. These would be the best methods to assess my technology plan because it is not the current plan being used by the district. The best thing is for my technology it to work closely with the current technology plan to get formally evaluated by the members of the current technology planning team.

3. Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.

I plan to collect data from the interviews, surveys, and from the STaR chart. I plan to use a variety of different information while collecting data. I plan to use the survey and interviews from teachers to determine the technology needs. These are the people that know exactly what is needed because they are working with the teachers. Often teachers point of view in not always considered when planning curriculum or technology. I also plan on working with the members of the technology planning team who is made up of a variety of people through out the district. Members include technology facilitators, and administrators.  The technology facilitator will be a greater resources because
 coordinator should assist in "developing and administering surveys, defining learning outcomes, and implementing evaluation rubrics throughout the school or district " (Frazier & Bailey, 2004). These people will help with the budget and planning. I am currently waiting to here back on the data of the  formative evaluation plan from the current technology plan.


References:

Dick & Carey (2001) Designing and Conducting Formative Evaluation. Retrieved (2012, April 9) from https://www.courses.psu.edu/trdev/trdev518_bow100/D_C10present/tsld001.htm

Frazier, M., & Bailey, G. D. (2004). The Technology Coordinator's Handbook. Eugene, OR: International Society for Technology in Education. Retrieved April 9, 2012, from Questia database: http://www.questia.com/PM.qst?a=o&d=113454356

Evaluation Toolbox (2010). Tool Selector. Retrieved April 9, 2012 from http://evaluationtoolbox.net.au/index.php?option=com_content&view=article&id=51&Itemid=131.

CFISD Kristina Coen

 
1. Describe the purposes for and various stages of formative evaluation of technology plan.

A formative evaluation is critical in order to test various aspects of instructional material in order to verify that the set goals are being meet. Therefore, improvements can be made changes and adjustments are still possible based upon the information gathered during the formative evaluation. Consider that your plan is still “forming” thus, having fresh set of eyes give input of the material will help polish and finalize your plan.

Dick and Carey have established four stages of the formative evaluation:

1.     Expert Review: Experts review a draft version of your material

2.     One to One: One learner at a time reviews the material with the evaluator. The evaluator makes notes as an observer.  

3.     Small Group: Similar to the One to One step except now the process is done within a controlled group setting.

4.     Field Test: The evaluator observes the instructional material in a realistic environment with learners.

At each stage expect changes to be made in order to ensure that the goals are reached. Whether if a technology plan is a created for a campus or an entire district it will involve a great deal of change for all of those it will encompass. Creating checkpoints in the form of formative evaluations prior to the final roll of the technology is essential for a smooth implementation of the new plan. A technology plan is your road map toward the 21^st century skill sets that we aim for yet if the directions we receive from our road map are unclear our distance will be a bumpy one.

2. Describe your instruments used in a formative evaluation.

Currently the only tool that is used regularly and district wide is the STaR Chart. Yet, I can truly believe that many teachers do not understand the weight that the data this survey collects can carry. I can say this with confidence because I distinctly remember my first year of teaching and just clicking randomly at the survey. No one told why I had to take it or what the information was for. It was just drilled repeatedly to take it or else.

There are several different types of instruments that will be used from example; I have listed classroom observations as part of the formative evaluation process. I believe for the data to be collected to be valid and useful a uniform rubric should be used. With a clearly written rubric both teachers and observers will share a common understanding of both goals and expectations. This is a sample rubric that will be used:

I also plan on using surveys periodically in order to create a baseline of data to compare progress or areas of weakness. Since the surveys will be tons more often they will not need to be as long as the STaR chart and will be customized to fit the exact goals of the district’s technology plan. I plan on using Google Docs to create and review data from these surveys. I know that other services exist but I am a great deal of positive experience using Google Docs and offers a great deal of flexibility for free I believe this is the ideal program to use. Sample of data collected using Google Docs (Forms). 

In addition interview will be conducted in order to expand on information gathered from the surveys and classroom observations. I believe that interviews should maintain a level of confidentiality in order to be able to gain the trust of those being interviewed. If the person being interview is fearful that anything negative or critical they may say during the interview will affect them during the summative evaluation many individuals will not speak freely. And the information gathered therefore will be tainted and useless.  

3. Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.

I was able to conduct a survey on a small group of administrators. Not all administrators had completed the survey at the time of this post therefore I will report on the information I have at hand.

• Eighty percent of those that survey reported that they agreed teachers needed more training and year round support in technology integration. 
• Yet only twenty-five percent of them felt that current technology was meeting the changing needs of the district. 
• All of those surveyed reported that they worried about the status of the budget over the next few years and the lasting effects it will have in the classroom.   

(Survey: Changes to Our Current Technology Plan, April 4, 2012)

References:

Dick, W., Carey, L., & Carey, J. O. (2001). The systematic design of instruction (5th ed.). New York: Longman.

Garrison, C., & Ehringhaus, M. (n.d.). Formative and Summative Assessment in the Classroom. Association for Middle Level Education - AMLE. Retrieved October 11, 2011, from http://www.amle.org/Publications/WebExclusive/Assessment/tabid/1120/Default.aspx 
Interviews are not supported in bibliographies by APA. Please cite it as an in-text citation.

Carol Taylor - Fairfield ISD Week 11 Step 8

1.       Describe the purposes for and various.
Roles of a Technology Coordinator/Facilitator

A district technology plan should focus on integrating technology into the teaching and learning process to transform the way teachers teach and students learn. 
A planning committee is critical to the success of any technology plan. The committee should include expertise in planning, building a vision, needs assessment, curriculum and instruction, evaluation, goal setting, professional development, technology hardware, support and integration, media/marketing, and financial planning.  

The areas of expertise are embedded in the five Technology Focus Areas (TFAs):

1.      Student Learning (includes technology skills)

2.      Teacher Preparation and Delivery of Instruction

3.      Administration / Data Management / Communication Processes

4.      Resource Distribution and Use

5.      Technical Support

(Frazier & Bailey, 2004, p. 6)



2.       Describe your instruments used in a formative evaluation.

In content terms, technology planning and policies should address three major areas: vision, access, and integration.  In terms of process, policies represent relative end states that begin with theadoption of a technology plan. This in turn involves a series of steps, ranging from

the determination of needs, the involvement of stakeholders, and the ratification

of a document, to the implementation, evaluation, and revision of the plan.  Assessing plans and policies involves evaluating the content of plans and documenting the existence of policies, as well as assessing the process of plan development and implementation.

TERM CATEGORIES
Major plan components  can include:

❚ review of technology status, needs assessment, and other pre-planning products

❚ vision/goal statements

❚ equity issues

❚ instructional uses of technology

INDICATORS

Pre-planning

phase completed or under way

Major plan components

Plan approval

Funding support

Stages of the pre-planning phase completed.

The major planning components are present.

Components of the plan.

The technology plan is approved.

Percentage of total technology plan budget that has funds committed to its support.

Percentage of the plan federally funded.

Percentage of the plan state funded.

Percentage of the plan funded through other (local or private)sources.14

TECHNOLOGY IN SCHOOLS

❚ student technology standards

❚ staff technology standards

❚ integration into core curriculum

❚ pilot program and action research

❚ management uses of technology

❚ student information systems

❚ infrastructure and support for infrastructure, includingsuch facilities-related needs as air conditioning/cooling and asbestos abatement

❚ review of current “state of the art” for options in design of infrastructure

❚ capabilities of hardware and software, projections of “next generation” capabilities and features

❚ acceptance and management of donated computing resources

❚ long-range goals

❚ inventory control issues, such as maintenance and replacement cycle

❚ budget projections and funding sources for initial installation, hardware, and software

❚ staff training programs

❚ benchmarking standards

❚ quality control components

❚ security planning

(Schmitt, 2002)

  

There are potential components of a technology plan that can overlap with a school or district’s facilities plan, such as network installation costs, including electrical wiring, maintenance and expansion. Technology planners and administrators will most likely want to decide in the pre-planning phase where to deal with facilities issues and related budgeting.

It is important to remember that technology or parts of the plan that are not implemented should not be considered failures. Implementing new technology can be a daunting undertaking and flexibility is needed for any change process. For this reason, evaluation in a variety of formats is critical in objectively determining what is working and what needs more attention.

  

INDICATORS

Evidence of evaluation

A review cycle (including timelines and reporting) is implemented.

There is a provision for revision of the plan.

The review is detailed in a report.

The report is readily available to the school and community.

The technology plan has been changed on the basis of the

most recent evaluation review.

Components of the review cycle.

The plan is achieving its goals.

As we embark on our journey to become the best technology facilitator, here are some websites that may be useful to you.

Planning tools

Guiding Questions for Technology Planning, Version 1.0, Regional Technology

Education Consortia (RTEC).

See  http://www.ncrtec.org/capacity/guidewww/basic.htm  and

http://www.ncrtec.org/capacity/guidewww/gqhome.htm

Learning through Technology, A Planning and Implementation Guide, North

Central Regional Educational Laboratory.

 See  http://www.ncrel.org/tandl/homepg.htm

(Schmitt, 2002)

3.   Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.
I found this formative evaluation on the following website  http://net.educause.edu/ir/library/html/EDU0240a.html
The components look to be very detailed in determining the efficacy of a well designed technology plan.  I think one of the greatest aspects of today’s technology is the opportunity to share information.

 The purpose of the rubric is to use by those who create the technology plan and administrators (Superintendents and Technology Coordinator) to reflect upon the plan to see if it meets the objectives set forth for the plan.  The evaluation should also involve the documentation of projected objectives to achieved within a certain time frame to determine if these objectives have been met and give explanation for the outcomes.  For our own district, we planned on providing 1 to 1 laptops for the students at the Intermediate, however, budget cuts and overspending did not allow for this goal to be achieved.  Therefore, a decision must be made when a new technology plan is created to determine if this objective will be continue to be a part of the plan or be removed.  This is one example why evaluations of a technology plan are important. “The technology coordinator should assist in developimg and administering surveys, defining learning outcomes, and implementing evaluation rubrics throughout the school or district. The information gathered from these assessments should be compiled and shared with stakeholders for further analysis and use in future planning” (Frazier & Bailey, 2004, p. 129).  The continual evaluation is essential when planning a new technology plan.   Barnet (2001) has developed 10 essentials for a successful technology plan:

1)      Create Vision

2)      Involve Stakeholders

3)      Gather Data

4)      Review the research

5)      Integrate technology into the curriculum

6)      Commit to professional development

7)      Ensure a sound infrastructure

8)      Allocate appropriate funding and budget

9)      Plan for ongoing assessment and monitoring

10)  Prepare for tomorrow

The following are survey results from our current technology plan.  I feel strongly that some of these responders have never even actually read the technology plan. 

https://docs.google.com/spreadsheet/gform?key=0AnhyZeEuyZ6sdFp1UXhqZTk0VmptU04zdGVxTWxISHc&gridId=0#chart

References

Baker, E. L. & O'Neil, H. F. (Eds.). (1994). Technology Assessment in Education and Training. Hillsdale, NJ: Lawrence Erlbaum Associates. Retrieved April 9, 2012, from Questia database: http://www.questia.com/PM.qst?a=o&d=13639211

Frazier, M., & Bailey, G. D. (2004). The Technology Coordinator's Handbook. Eugene, OR: International Society for Technology in Education. Retrieved April 9, 2012, from Questia database: http://www.questia.com/PM.qst?a=o&d=113454356

Oliver, J. (1997, April). 10 'Must Ask' Questions When Developing a Technology Plan.School Administrator, 54, 26. Retrieved April 9, 2012, from Questia database: http://www.questia.com/PM.qst?a=o&d=5001627655

SchS    Schmitt, C. (2002, November). NCES—Technology In Schools. Retrieved from National Center for Education Statistics: http://nces.ed.gov/pubs2003/2003313.pdf

Middle School - Laura Willcox - Week 11

1.  Describe the purposes for and various stages of formative evaluation of technology plan.

Formative Evaluation: The process of collecting data that can be used to revise the instruction before implementation thus making the instruction more effective. A pilot test is an example of Formative Evaluation. (Dick & Carey Model Glossary n.d.).

The three types of formative evaluation are: (1) one-to-one evaluation, (2) small-group evaluation, and (3) field evaluation. Conducting a series of formative evaluations resulting in data collection will identify areas of instruction in need of revision.

Once a prototype of the instructional materials is created, they should be piloted with a group of learners to collect data in order to identify how the instruction can be improved.  (Dick & Carey - Develop & Conduct Formative Evaluation. n.d.)

2.  Describe your instruments used in a formative evaluation.

The technology plan is based on a formative evaluation. I will use the data collected for the Star Chart as my reevaluation as all the teachers on my campus has already completed it and it is a standard tool used across the state. I also think it is important to keep notes from meetings, interviews and conversations having to do with developing the technology plan.

3. Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation.

I will use the data from the Star Chart and interviews from the teachers to collect data. The teachers are the ones in the classroom and who would know best what is needed in terms of technology. They are the ones that work with the students daily and know their needs better than anyone. Next, the technology director would know what is needed in terms of budgeting. Sitting down with him and determining the needs there would be the next step. Then, working with management to build the bridge between needs and money would be the last step. The teachers know the needs, the technology director would know the budget and management would know where to get the money from.

Dick & Carey - Develop & Conduct Formative Evaluation. (n.d.). College of Education | Idaho State University. Retrieved April 9, 2012, from http://ed.isu.edu/depts/imt/isdmodels/Dick/Formative_Eval/Dick_Formative_desc.html

Dick & Carey Model Glossary. (n.d.). College of Education | Idaho State University. Retrieved April 9, 2012, from http://ed.isu.edu/depts/imt/isdmodels/Dick/glossary.html

Week 11 Step 8 - Searight

Week 11 Step 8 - Revise Technology Plan / Design and Conduct Summative Evaluation

1.   Describe the purposes for and various stages of formative evaluation of technology plan.

     Technology Plan

Goals:

     Improve Student Learning with technology resources

Improve student learning through instructional technology.

Improve institutional decision-making

Improve decision making by increasing access to valid and reliable institutional data.

Improve Instructional and Administrative technology access

Improve access to software, hardware, and network

resources.

Improve technology customer support systems

Provide 24/7 technology support for the McLennan Community College user community. Develop online support and professional development resources for student, faculty, and staff.

The Information Systems and Services Department works with the Research and Planning Department to develop student and faculty/staff technology survey’s to measure user satisfaction.  We also utilize this medium to obtain user input on specific topics.  As the Director, I’ll meet one-on-one with the President, the Vice President’s and various department heads to get feedback related to instructional and administrative technology.  I also try very hard to meet with as many faculty members as I can.  The ISS department will also meet with departments or small groups of employees to get feedback or introduce new solutions.  Within the Information Technology realm we often develop “pilot” projects for users to evaluate and test within their environments.  Therefore, formative evaluation is an ongoing effort that’s very time consuming but rewarding if done right!  Throughout these processes, it’s very important to refer to the overall campus technology plan.

2.   Describe your instruments used in a formative evaluation.

Survey Monkey, Google Forms/Surveys, and a homegrown web based solution.  We also utilize SharePoint 2010 to share information related to information technology, projects, budgets, solutions, etc…   I also think it’s very important to always take meeting notes and develop action items to follow-up on.  I’ll utilize IssueTrak and SharePoint 2010 to store information related to meetings and projects.

3.   Collect data according to a formative evaluation plan for a given set of technology plan or instructor presentation. 

     Survey Examples:

     2010 Professional Development Survey

     2010 Student Satisfaction Survey

References:

Illich, P. (2012, January). Mcc's office of research planning

    and institutional effectiveness. Retrieved from

    http://mcciep.mclennan.edu/