Home » Projects » Examining the Relationship Between Qualitative Indices...

Overview

This report examines the relationship between qualitative indices in planning documents and real-world indicators such as geographic and socioeconomic characteristics. The four qualitative indices analyzed include:

Inclusiveness: The extent to which diverse stakeholders are engaged in planning processes.
Scientific Basis: The degree to which decisions are supported by empirical research and evidence.
Viability and Structure: The clarity, feasibility, and internal coherence of planning strategies.
Regional Cooperation: The extent of inter-municipal or cross-regional coordination in planning efforts.

Previous Literature

As mentioned during our last meeting, Yuanshuo is also working on a similar topic. The proposed methodology is to measure the equity language used in planning document. Each document is ranked from 1 to 3, where 3 is the highest (most equitable) and 1 is the least. However, compared to the measurement used in this study, they are measuring each document as a whole, while we are measuring certain characteristics’ presence in the document by chunks/parts.

Methodology

Analytical Framework

Large Language Model: Utilizing a pre-trained, general purpose language model to extract qualitative indices from planning documents. Each category is prompted with a specific description and several keywords to guide the model’s attention.
Measurement: Based on the percentage of chunks that contains languages that are categorized by the model as one of the four qualitative indices, each document is assigned a score from 0 to 1 (Density) for each index. The average density is calculated for each planning region to represent the region’s planning document quality in terms of the indices.

Prompts and Keywords:

Inclusiveness:

  "TOPIC 13": "Inclusiveness",
    "TOPIC 13 Explaination": "Inclusive urban planning for all social groups and cooperation with the private sector",
    "TOPIC 13 Keywords": "Participatory process, stakeholder engagement, private sector cooperation"

Scientific Basis:

"TOPIC 14": "Scientific Basis",
    "TOPIC 14 Explaination": "Scientific basis for climate change adaptation and mitigation measures",
    "TOPIC 14 Keywords": "Climate models, climate projections, climate scenarios, climate data, climate change impacts, climate change adaptation, climate change mitigation"

Viability and Structure:

"TOPIC 15": "Viability and Structure"
"TOPIC 15 Explaination": "Viability and structure of climate change adaptation and mitigation measures"    
"TOPIC 15 Keywords": "Cost-benefit analysis, feasibility study, implementation plan, monitoring and evaluation, risk assessment, risk management, financing mechanisms, funding sources, public-private partnerships, international cooperation"

Regional Cooperation:

 "TOPIC 16": "Regional Cooperation",
    "TOPIC 16 Explaination": "Regional cooperation, and Federal/Regional level cooperation",
    "TOPIC 16 Keywords": "Inter-municipal cooperation, regional cooperation, cross-border cooperation, federal level cooperation, regional level cooperation, international cooperation, transnational cooperation"

Correlation Analysis: The relationship between the qualitative indices and real-world indicators is examined through correlation and regression analysis.

Results

Spatial Results

Figure 1. Spatial Distribution of Inclusiveness (Click to Expand)

Figure 2. Spatial Distribution of Scientific Basis (Click to Expand)

Figure 3. Spatial Distribution of Viability and Structure (Click to Expand)

Figure 4. Spatial Distribution of Regional Cooperation (Click to Expand)

Correlation & Regression Analysis

This part’s real-world variables are different from the latest version, as they are from the previous analysis. They are not strictly grouped by categories (urban structure, economic, etc.), and the data is from the latest year.

With regard of the four indices on planning languages, please focus on the last four rows.

Figure 5. Correlation Matrix with Demographic Indicators (Click to Expand)

Figure 6. Correlation Matrix with Economic Indicators (Click to Expand)

Figure 7. Regression Analysis with Political Indicators (Click to Expand)

Interpretation of Findings

Spatial Distribution

While Regional Cooperation seems to be more prominent across documents, Scientific Basis and Viability and Structure are less frequently mentioned. Though these indicators presents a largely patchy pattern, the four indices, especially the ones that are less frequently mentioned, are more likely to be found in the same region, revealing a potential correlation in incorporating more inclusive/structured language in planning documents.

Specifically, Northern Germany stands out for having a higher density of Regional Cooperation language, possibly due to cross-boundary issues like coastal management (which, is also reflected in the later correlation analysis) or shared economic interests. The sourthern and central band regions also exhibit a higher density of Regional Cooperation, overlapping with larger metropolitan areas. These areas spanning multple municipalities may require more joint strategies for infrastructure or economic development.

However, the spatial distribution of other indices is less clear. These topics, considered less essential in planning documents and only gained increased attention in recent years, are less likely to be mentioned in the same document.

Economic Indicators

Tourism (Tourism capacity, Foreign Guests)

There’s a strong and positive relationship with Inclusiveness and moderate positive correlation with Regional Cooperation. Maybe tourism sector requires inclusive decision-making process to balance the need between resident quality of life and visitor infrastructure? Also, Tourism planning might require coordinated planning, for example cross-municipal development and recreation corridors.

It’s worth noting that there is a strong, negative correlation between Foreign Guests and Regionalk Cooperation. A likely explaination is that regions hosting large numbers of foreign guests (as illustrated in the main report) - major urban centers or tourist hubs - tend to operate relatively independently rather than cooperating with other regions. For instance, major city-regions with already established strong infrastructure reduce their reliance on neighboring regions. It’s also possible that the lack of regional planning documents in these major municipalities contributes to the negative correlation.

Economic Development and Industry Sectors

Higher share of Primary and Tertiary Sectors seems to positively correlated with Inclusive, suggesting that regions with heavy industry and resource extraction are more often to lean away from inclusiveness, as well as other indices. Higher economic output may lead to more inclusive decision-making processes due to the increased awareness in the population. The slight negative correlation between Regional Cooperation and Teritary Sector might be due to the fact that regions with a higher share of tertiary sector are more likely to be self-sufficient and less reliant on cooperation with other regions.

Demographic Indicators

Population Density and Urban Structure

More densely populated areas shows a modest positive correlation with Inclusiveness, Viability and Structure, and Scientific Basis, suggesting that areas of urban clusters are more likely to incorporate the aforementioned language in their planning documents. This could be due to the increased complexity of urban planning in these areas, requiring more structured and evidence-based decision-making processes, or the increasing awareness of certain approaches in urban planning. As discussed before, these urban centers are more unlikely to have regional cooperations in their planning documents, as they are more self-sufficient and less reliant on cooperation with other regions.

Water Area, correlates with coastal tourism regions, shows strongest positive correlations for Inclusiveness. Forest Area, however, may correlate with secondary sector, shows an overall negative correlation with all four indices. This might also be due to the fact that regions with a higher share of forest area are more likely to be rural and less densely populated, leading to less structured planning documents.

Waste and Recycling

Household/Bulky waste and recyclables are regarded as indicators of sustainable development. However, the correlation with the four indices are counterintuitive. Places with more recyclables are less likely to incorporate the four indices in their planning documents, while our initial hypothesis suggests that higher awareness in sustainable development should lead to more structured, inclusive, and evidence-based planning documents.

Political Indicators

Overall, the parties’ philosophy does not clearly translate into regional planning languages. Factors such as temporal vote‐share changes, preexisting federal frameworks, and the specific climate risks regions face can all shape how (and whether) inclusiveness, scientific evidence, clear feasibility, or cross‐municipal alliances appear in formal planning documents.

Inclusiveness, Scientific Basis, and Viability are positively correlated with left-leaning parties (SPD, Die Linke), likely reflecting their ideological emphasis on collective discussion making processes. However, there’s a surprising negative correlation with GRÜNE. FDP’s strong negative correlation is possible due to its emphasis on efficiency and limited government intervention, as well as seen in other indicators.
Regional Cooperation has a counterintuitive positive correlation with AfD, as the party generally opposes interregional collaboration. This is likely due to the geographical confounders, where AfD has a strong support in eastern German regions with historical cross-border planning importance? Die Linke’s positive correlation aligns with the party’s emphasis on collective planning approaches. CDU’s strong negative correlation might reflect the traditional value on local autonomy, while other correlations seems to be modest and without significance.

Concerns and Limitations

Methodological Limitations

Lack of confirmation and ground-truth validation

Compared to Yuanshuo’s approach in measuring the equity language in planning documents, our methodology relied on a one-shot prediction framework with minimal prompt engineering and no iterative training procedures.

While Yuanshuo’s team employed a class of urban planning students to manually evaluate approximately 10% of documents and created a validation set for further tuning, our analysis lacks this verification component. Their approach also benefited from a structured measurement framework with weighted criteria for assessing equity considerations (e.g., 20% for procedural equity, 30% for implementation outcomes, etc.). This framework also had multiple rounds of reworking and prompt refinement based on manual validation findings. The lack of such validation in our study potentially introduces measurement bias into our results.

References

TBD

Examining the Relationship Between Qualitative Indices in Planning Documents and Real-World Indicators