A methodology designed for understanding and improvement
This study examines how Chicago Public Schools dual-language schools protect program integrity and equity for multilingual learners amid competing internal and external pressures. It also examines how the Office of Multilingual and Multicultural Education, network structures, and related district systems support or constrain that work.
The methodology is designed to produce findings that are credible, contextually grounded, and usable by the leaders responsible for sustaining high-quality dual-language programming.
Research Design Overview
The study uses a utilization-focused, improvement-oriented, qualitative-dominant mixed-methods program evaluation conducted through naturalistic inquiry.
Utilization-focused evaluation organizes the study around the information needs and practical decisions of clearly identified intended users. The purpose is not simply to produce findings, but to generate knowledge that can inform decisions, strengthen implementation, and support development of the Dual-Language Program Integrity Framework.
A naturalistic design is appropriate because dual-language implementation is shaped by real organizational settings, local relationships, policy demands, staffing conditions, governance structures, and leadership decisions. These dynamics cannot be fully understood by examining the program outside the contexts in which educators and leaders experience them.
Qualitative evidence provides the primary source of explanation and meaning. Limited descriptive quantitative evidence is used to contextualize cases, illuminate patterns, and complement—not replace—the perspectives and experiences of participants.
Utilization-Focused
Evaluation questions and procedures are organized around the information needs of leaders who can use the findings to improve policy, support, monitoring, and decision-making.
Improvement-Oriented
The evaluation seeks to identify strengths, gaps, recurring pressures, and practical opportunities for strengthening dual-language implementation and equity.
Naturalistic
Program implementation is studied within the real environments in which school and district leaders interpret policies, negotiate constraints, and make decisions.
Primary Intended Users
The study is designed around the decision-making needs of leaders who influence dual-language program implementation, support, oversight, and accountability across CPS.
School Leaders
Principals and assistant principals make daily decisions about staffing, scheduling, language allocation, student access, instructional priorities, services, and family communication.
OMME Leaders and Staff
District multilingual leaders develop guidance, professional learning, program supports, monitoring practices, and implementation expectations for dual-language schools.
Network Leaders
Network chiefs, deputy chiefs, and support personnel influence principal supervision, implementation assistance, resource coordination, accountability messaging, and escalation.
District Decision-Makers
Central-office departments shape staffing systems, enrollment policies, budgets, curriculum, special education services, accountability expectations, and other conditions affecting program integrity.
The usefulness of the evaluation depends on whether its findings can help leaders make more coherent, equitable decisions under real-world conditions.
Four Arenas of Change
Wagner and Kegan’s four arenas of change organize data collection, coding, interpretation, and framework development. The arenas make it possible to examine dual-language implementation as an interconnected system rather than as a series of isolated technical decisions.
Contexts
External policy, political, economic, accountability, enrollment, and governance forces that influence implementation.
Culture
Shared beliefs, language ideologies, expectations, norms, and assumptions about bilingualism, equity, quality, and belonging.
Conditions
Staffing, schedules, curriculum, materials, services, resources, structures, routines, guidance, and accountability systems.
Competencies
The technical, instructional, relational, political, and leadership knowledge and skills required to protect program integrity and equity.
The four arenas help reveal whether implementation challenges arise primarily from external pressures, organizational beliefs, structural conditions, leadership capacity, or interactions among all four.
Participants and System Perspectives
The study brings together participants positioned at different levels of the CPS dual-language system.
Participants are selected because their roles provide direct knowledge of how dual-language programs are designed, implemented, supported, monitored, adapted, and protected under conditions of constraint.
Principals and Assistant Principals
School leaders provide accounts of day-to-day implementation, staffing and scheduling decisions, equity dilemmas, governance pressures, program monitoring, and interactions with district and network structures.
OMME Staff and Leaders
Central-office participants explain the district’s intended theory of action, formal guidance, professional learning, monitoring routines, tools, authority, coordination structures, and implementation challenges.
Network-Level Leaders
Network leaders contribute insight into principal support, supervisory expectations, resource coordination, accountability, escalation, and the conditions affecting implementation across multiple schools.
The study focuses on adult professionals and organizational systems. Students are not recruited or directly observed, and no new identifiable student-level data are collected.
Purposeful and Maximum-Variation Sampling
The study uses purposeful, criterion-based sampling to identify information-rich participants whose professional responsibilities relate directly to dual-language program implementation, leadership, support, oversight, or accountability.
Maximum-variation considerations are used to capture meaningful differences in program and organizational context. These differences may include program model, grade configuration, school type, geographic area, program maturity, student demographics, and the participant’s system-level role.
-
1
Establish eligibility
Publicly available district and school information is used to identify eligible dual-language schools, current school leaders, and relevant district or network roles.
-
2
Seek meaningful variation
Recruitment seeks perspectives from different program models, school settings, leadership roles, and implementation conditions.
-
3
Recruit in phased waves
Invitations are sent in stages so later recruitment can address important representation gaps identified during earlier waves.
-
4
Assess informational sufficiency
Final sample size is guided by the study’s analytic needs and by whether additional interviews continue to produce new insights or primarily reinforce established patterns.
Multiple Sources of Evidence
The evaluation uses multiple data sources to capture formal program expectations, lived implementation experiences, organizational decision-making, and descriptive patterns across participating contexts.
Semi-Structured Interviews
Interviews provide detailed accounts of implementation conditions, leadership decisions, equity dilemmas, district and network supports, governance dynamics, non-negotiables, adaptations, indicators, and safeguards.
Document Analysis
Policies, guidance, tools, schedules, program materials, professional learning resources, monitoring artifacts, and public governance records are examined systematically.
Optional Administrator Survey
A brief survey may be used to validate and prioritize emerging framework components and to examine patterns in perceived importance, feasibility, and clarity.
Limited Adult-Meeting Observations
Where feasible and approved, non-intrusive observations of adult professional meetings may provide insight into how decisions, tools, tensions, and implementation priorities operate in practice.
Descriptive Indicators
Aggregate school-level information may be used to describe program contexts and illuminate patterns related to enrollment, participation, staffing, access, and publicly reported outcomes.
Analytic Memos
Reflexive and analytic memos document emerging interpretations, questions, relationships among themes, and decisions made throughout the research process.
Semi-structured interviews serve as the primary data collection method because they allow consistent inquiry across participant groups while preserving flexibility to explore role-specific experiences and unexpected themes.
Document and Artifact Analysis
Documents make it possible to compare the district’s formal expectations with the realities participants describe.
Document analysis focuses on non-confidential program and implementation materials. Documents are reviewed for evidence related to program design, equity, roles and responsibilities, monitoring, decision-making, implementation supports, and the response expected when program integrity is threatened.
Student education records, student identifiers, individual IEP or 504 documents, discipline records, individual case notes, staff evaluations, and confidential personnel records are excluded from document collection and analysis.
Data Analysis
Qualitative and descriptive quantitative evidence are analyzed in parallel and then brought together during interpretation. The analysis is iterative, meaning that coding, memoing, comparison, and refinement occur throughout the study rather than only after all data collection has ended.
-
1
Prepare and organize the evidence
Interview transcripts, notes, documents, observation records, survey results, and descriptive indicators are organized using study identifiers and secure data-management procedures.
-
2
First-cycle coding
Descriptive, in vivo, and process codes capture participants’ language, central concepts, significant actions, pressures, and experiences related to the four arenas of change.
-
3
Second-cycle coding
Pattern coding groups initial codes into broader categories and themes that address the research questions and emerging integrity framework.
-
4
Within-case analysis
Evidence is synthesized within each school or organizational context to describe implementation conditions, pressures, leadership responses, and integrity-related patterns.
-
5
Cross-case analysis
Cross-case matrices and thematic comparisons identify common patterns, meaningful differences, contradictions, and role-specific perspectives across schools and system levels.
-
6
Framework synthesis
Findings are translated into proposed non-negotiables, allowable adaptations, red-line thresholds, decision rules, indicators, safeguards, and escalation pathways.
Primary Coding Lenses
Mixed-Methods Integration and Triangulation
Integration occurs during both analysis and interpretation. Themes from interviews and documents are compared with observations, survey responses where used, and descriptive program indicators.
Triangulation
Multiple sources are compared to determine where evidence converges, where it differs, and where one source complicates or clarifies another.
Complementarity
Descriptive indicators help illuminate the organizational and equity context surrounding participants’ qualitative accounts.
Development
Preliminary findings may inform follow-up questions, document requests, survey items, and refinement of emerging framework components.
The analysis does not treat disagreement among participants or data sources as a problem to be removed. Contradictions, disconfirming evidence, and rival explanations are examined because they may reveal important differences in authority, experience, beliefs, or implementation conditions.
Credibility is strengthened when conclusions are tested across multiple roles, settings, documents, and forms of evidence.
Ethical Safeguards
The study is designed to minimize professional risk, protect confidentiality, preserve voluntary participation, and limit data collection to what is necessary.
Informed Consent
Participants receive study information before data collection and may ask questions, decline participation, skip questions, pause, or withdraw without penalty.
Optional Audio Recording
Audio recording requires separate permission. Participants may complete an interview without being recorded, in which case detailed notes are used.
Confidentiality
Pseudonyms, generalized role descriptors, removal of direct identifiers, and careful reporting practices are used to reduce the risk of deductive disclosure.
Data Minimization
The study collects only information necessary to address the research questions and avoids unnecessary identifying or professionally sensitive details.
Secure Storage
Digital research materials are stored using encrypted, password-protected systems, with identity keys maintained separately from research data.
Adult-Only Research Activities
Interviews, surveys, and observations involve adult professionals. No classroom observation or direct research activity with students is conducted.
Researcher Positionality and Reflexivity
The researcher’s perspective is shaped by experience as a bilingual learner, teacher, principal of a CPS dual-language school, and district-level leader supporting multilingual learners.
This practitioner experience provides knowledge of dual-language program design, school leadership, district structures, and the practical dilemmas leaders encounter. It also creates a responsibility to examine how prior experiences, assumptions, and professional relationships may influence data collection and interpretation.
Reflexive memoing, comparison across participant groups, document analysis, attention to disconfirming evidence, consultation with the dissertation chair, and transparent documentation of analytic decisions are used to strengthen credibility and reduce the risk that conclusions reflect only the researcher’s prior experiences.
Practitioner knowledge is treated as a source of insight that must also be examined critically through reflexivity, triangulation, and disciplined analysis.
Study Limitations
The study is designed to provide deep, contextually grounded understanding rather than statistical generalization or causal claims.
Purposeful Sample
Participants are selected for their relevant knowledge and experience rather than through random sampling. Findings therefore support analytic transferability, not statistical representativeness.
Single-District Context
The study is bounded to one large urban district and a specific period in its policy, leadership, staffing, and program environment.
Reliance on Self-Report
Interview and survey responses may be influenced by memory, professional position, social desirability, or differences in how participants interpret events.
Modest Sample Size
The in-depth design limits the number of schools and system actors that can be included and may leave some relevant perspectives outside the study.
Deductive Disclosure Risk
Leadership roles may remain recognizable even when names are removed, requiring careful use of quotations, descriptors, and contextual details.
Researcher Influence
The researcher’s professional background may shape attention and interpretation, making reflexive and triangulated analytic practices essential.
Rich descriptions of contexts, pressures, organizational conditions, and leadership practices allow readers to judge whether the findings are relevant to other dual-language schools, multilingual offices, or urban school systems.
From Evidence to an Actionable Framework
The methodology connects multiple system perspectives, forms of evidence, analytic procedures, and ethical safeguards to examine how dual-language program integrity and equity are enacted in practice.
Together, these methodological choices are intended to generate credible findings that explain how leaders protect—or struggle to protect—dual-language program integrity amid competing pressures.
The resulting analysis will inform the identification of program non-negotiables, allowable adaptations, decision rules, practical indicators, monitoring routines, district safeguards, and escalation pathways.
Methodology in Service of Improvement
The study is not designed to evaluate the performance of individual employees or schools. It is designed to support system-level learning by examining how policies, beliefs, structures, resources, leadership practices, and governance conditions interact.
Its central methodological commitment is to produce trustworthy evidence that can be translated into practical guidance for protecting dual-language program integrity and advancing equity for multilingual learners.