This notebook estimates the indicators based on the raw clean data and perfomrs the main analyses and figures used in the manuscript of the multicountry paper. The input is the "clean kobo output" that was first cleaned by `2_cleaning.Rmd`. Besides the plots and statistics shown below, the output is are the indicators data ready to be used to estimate the indicators (`ind[1-3]_data.csv`), a single file with the indicators already calculated (`indicators_full.csv`) and metadata.

## Packages and functions

Load required libraries:

```{r, warning=FALSE, message=FALSE}
library(tidyr)
library(dplyr)
library(readr)
library(utile.tools)
library(stringr)
library(ggplot2)
library(ggsankey)
library(ggnewscale)
library(alluvial)
library(viridis)
library(cowplot)
library(lme4)
library(knitr)
library(glmmTMB)
```

Load required functions. These custom fuctions are available at: https://github.com/AliciaMstt/GeneticIndicators

```{r source}
source("get_indicator1_data.R")
source("get_indicator2_data.R")
source("get_indicator3_data.R")
source("get_metadata.R")
source("transform_to_Ne.R")
source("estimate_indicator1.R")
```

Other custom functions:
```{r custom_funs}
### not in
'%!in%' <- function(x,y)!('%in%'(x,y))


#' Duplicates data to create additional facet. Thanks to https://stackoverflow.com/questions/18933575/easily-add-an-all-facet-to-facet-wrap-in-ggplot2
#' @param df a dataframe
#' @param col the name of facet column
#'  
CreateAllFacet <- function(df, col){
  df$facet <- df[[col]]
  temp <- df
  temp$facet <- "all"
  merged <-rbind(temp, df)

  # ensure the facet value is a factor
  merged[[col]] <- as.factor(merged[[col]])

  return(merged)
}
```

Custom colors:
```{r custom colors}
## IUCN official colors
# Assuming order of levels is: "re", "cr", "en", "vu", "nt", "lc", "dd", "not_assessed", "unknown" (for regional, and w/o "re" for global). Make sure to change the levels to that order before plotting. Note: NE should be "#FFFFFF" but since that means white I'm suing azure2
IUCNcolors<-c("#FF0000", "#FFA500", "#FFFF00", "#ADFF2F", "#008000", "#808080", "azure2", "bisque1")
IUCNcolors_regional<-c("darkorchid2", "#FF0000", "#FFA500", "#FFFF00", "#ADFF2F", "#008000", "#808080", "azure2", "bisque1")

## nice soft ramp for taxonomic groups
taxoncolors<-cividis(12) # same than using cividis(length(levels(as.factor(metadata$taxonomic_group))))

## Colors for simplified methods to define populations 
# assuming the levels (see how this was created in the section "Simplify combinations of methods to define populations"): of running levels(as.factor(ind2_data$defined_populations_simplified)) (after new order)

# get a set of colors to highlight genetic and geographic with similar colors

simplifiedmethods_colors<-c("#FFA07A", #"dispersal_buffer"
                            "#7f611b", # "eco_biogeo_proxies"
                            "#668cd1", # "genetic_clusters"     
                            "#668cd1", # "genetic_clusters eco_biogeo_proxies"     
                            "#45c097", # "genetic_clusters geographic_boundaries"  
                            "#d4b43e", # "geographic_boundaries"
                            "#d4b43e", # "geographic_boundaries eco_biogeo_proxies"
                            "#d4b43e", # "geographic_boundaries management_units" 
                            "#b34656", # "management_units" 
                            "#be72c9", # "other" 
                            "#be72c9")# "other_combinations" 

grouped_taxon_colors<-c("#a46cb7", "#7aa457", "#cb6a49")
  
```

## Get data 

Get indicators and metadata data from clean kobo output
```{r, echo=TRUE}
# Get data:
kobo_clean<-read.csv(file="kobo_output_clean.csv", header=TRUE)

# Extract indicator 1 data from kobo output, show most relevant columns
ind1_data<-get_indicator1_data(kobo_output=kobo_clean)
head(ind1_data[,c(1:3, 12:14)])

# Extract Proportion of maintained populations (indicator) data from kobo output, show most relevant columns
ind2_data<-get_indicator2_data(kobo_output=kobo_clean)
head(ind2_data[,c(1:3, 9:10,13)])

# Extract indicator 3 data from kobo output, show most relevant columns
ind3_data<-get_indicator3_data(kobo_output=kobo_clean)
head(ind3_data[,c(1:3, 9:11)])

# extract metadata, show most relevant columns
metadata<-get_metadata(kobo_output=kobo_clean)
head(metadata[,c(1:3, 12, 25,26, 64)])
```

Get population data for those species assessed using the tabular text template instead of Kobo. This file was produced by the script `1.2_cleaning.Rmd`
```{r}
ind1_data_from_templates<-read.csv(file="ind1_data_from_templates.csv")

```

Add data recorded using the population template to the ind1_data already in the nice format.
```{r}
ind1_data<-rbind(ind1_data, ind1_data_from_templates)
```


## Estimate indicators

### Indicator 1 or Ne 500 indicator (proportion of populations with Ne >500):

Show most relevant columns of indicator 1 data
```{r}
ind1_data %>% select(country_assessment, taxon, population, Name, Ne, NeLower, NeUpper, NeYear, GeneticMarkers, NcType, NcMethod, NcRange)

```

Remember what the function to transform NcRange and NcPoint data into Ne does:
```{r}
# check what the custom funciton does
transform_to_Ne
```

Use function to get Ne data from NcRange or NcPoint data, and their combination (Ne estimated from Ne if Ne is available, otherwise, from Nc)

```{r}
ind1_data<-transform_to_Ne(ind1_data = ind1_data, ratio = 0.1)
```

Check transformation in example:
```{r}
ind1_data %>% select(country_assessment, taxon, population, Name, Ne, NeLower, NeUpper, NeYear, GeneticMarkers, NcType, NcMethod, NcRange, Nc_from_range, Ne_from_Nc, Ne_combined)
```


Remember what the function to estimate indicator 1 does:
```{r}
# check what the custom function does
estimate_indicator1
```

**Now estimate indicator 1 :)**

```{r}
indicator1<-estimate_indicator1(ind1_data = ind1_data)
```

Check example data:
```{r}
indicator1 %>% select(X_uuid, taxon, country_assessment, n_pops, n_pops_Ne_data, n_pops_more_500, indicator1)
```


### Indicator 2 or PM indicator: proportion of populations within species which are maintained.

Show most relevant columns of indicator 2 data:
```{r}
ind2_data %>% select(country_assessment, taxon, n_extant_populations, n_extint_populations)

```


Proportion of maintained populations (indicator) is the he proportion of populations within species which are maintained. This can be estimated based on the `n_extant_populations` and `n_extint_populations`, as follows:

```{r}
ind2_data$indicator2<- ind2_data$n_extant_populations / (ind2_data$n_extant_populations + ind2_data$n_extint_populations)
```

Example output selecting the most relevant columns:

```{r}
ind2_data %>% select(country_assessment, taxon, n_extant_populations, n_extint_populations, indicator2)
```


### Indicator 3 or genetic monitoring indicator: number of species with genetic diversity monitoring

Example of indicator 3 data selecting the most relevant columns:
```{r}
ind3_data %>% select(country_assessment, taxon, multiassessment, temp_gen_monitoring, gen_studies, gen_monitoring_years, source_genetic_studies)
```



Indicator 3 refers to the number (count) of taxa by country in which genetic monitoring is occurring. This is stored in the variable `temp_gen_monitoring` as a "yes/no" answer for each taxon, so to estimate the indicator, we only need to count how many said "yes", keeping only one of the records when the taxon was multiassessed.

```{r}
indicator3<-ind3_data %>%
                 # keep only one record if the taxon was assessed more than once within the country
                 select(country_assessment, taxon, temp_gen_monitoring) %>%
                 filter(!duplicated(.)) %>%

                 # count "yes" in tem_gen_monitoring by country
                 filter(temp_gen_monitoring=="yes") %>%
                 group_by(country_assessment) %>%
                 summarise(n_taxon_gen_monitoring= n())
```

Example output:

```{r}
indicator3
```



### Join indicators and metadata in a single table

It could be useful to have the PM and Ne 500 indicators and the metadata in a single large table.
```{r}
indicators_full<-left_join(metadata, indicator1) %>% 
                     left_join(ind2_data)
```

### Check number of rows of each object, i.e. n including alternative assessments
```{r}
# Number of populations inc alternative assessments
nrow(ind1_data)

# Number of assessments including multiassessments (these 3 objects should have the same number of rows)
nrow(indicators_full)
nrow(ind2_data)
nrow(ind3_data)
nrow(metadata)


```

### Save indicators data
Save indicators data and metadata to csv files, useful for analyses outside R.

```{r}
# save processed data
write.csv(ind1_data, "ind1_data.csv", row.names = FALSE)
write.csv(indicators_full, "indicators_full.csv", row.names = FALSE)
write.csv(ind2_data, "ind2_data.csv", row.names = FALSE)
write.csv(ind3_data, "ind3_data.csv", row.names = FALSE)
write.csv(metadata, "metadata.csv", row.names = FALSE)

```


## Change country name to nicer labels

To have nice levels in the plots we will change the way country names are written: 

```{r nice country names}
# make factor
metadata$country_assessment<-as.factor(metadata$country_assessment)
indicators_full$country_assessment<-as.factor(indicators_full$country_assessment)
ind2_data$country_assessment<-as.factor(ind2_data$country_assessment)
ind1_data$country_assessment<-as.factor(ind1_data$country_assessment)
indicator1$country_assessment<-as.factor(indicator1$country_assessment)

# original levels
levels(metadata$country_assessment)

# change
levels(metadata$country_assessment)<-c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA")
levels(indicators_full$country_assessment)<-c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA")
levels(ind1_data$country_assessment)<-c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA")
levels(ind2_data$country_assessment)<-c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA")
levels(indicator1$country_assessment)<-c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA")


```


## Simplify combinations of methods to define populations

The methods used to define populations come from a check box question were one or more of the following categories can be selected: genetic_clusters, geographic_boundaries, eco_biogeo_proxies, adaptive_traits, management_units, other. As a consequence any combination of the former can be possible. Leading to the following frequency table:

```{r}
table(indicators_full$defined_populations)
```

It is hard to group the above methods, so we will keep the original groups with n >=19 in the above list, and tag the combinations that appear few times as as "other_combinations".

Which groups have n>=19?
```{r}
x<-as.data.frame(table(indicators_full$defined_populations)[table(indicators_full$defined_populations) >= 19])
colnames(x)[1]<-"method"

x

```

We can add this new column to the metadata and indicator data:

```{r} 
### for indicators 
indicators_full<- indicators_full %>% 
  mutate(defined_populations_simplified = case_when(
         # if the method is in the list of methods n>=19 then keep it
         defined_populations %in% x$method ~ defined_populations,
         TRUE ~ "other_combinations"))


### for meta
metadata<- metadata %>% 
  mutate(defined_populations_simplified = case_when(
         # if the method is in the list of methods n>=19 then keep it
         defined_populations %in% x$method ~ defined_populations,
         TRUE ~ "other_combinations"))

### for ind1 raw data
ind1_data<- ind1_data %>% 
  mutate(defined_populations_simplified = case_when(
         # if the method is in the list of methods n>=19 then keep it
         defined_populations %in% x$method ~ defined_populations,
         TRUE ~ "other_combinations"))
         
```


Check n for simplified methods:

```{r}
table(indicators_full$defined_populations_simplified)
```

Table of equivalences:

```{r}
indicators_full %>% 
       select(defined_populations, defined_populations_simplified) %>% 
       filter(!duplicated(defined_populations))
```

Create nicer names for ploting
```{r}
# original method names
levels(as.factor(indicators_full$defined_populations_simplified))


# nicer names
nice_names <- c("dispersal buffer",
                "eco- biogeographic proxies",
                 "genetic clusters",
                 "genetic clusters & eco- biogeographic proxies",
                 "genetic clusters & geographic boundaries",
                 "geographic boundaries",
                 "geographic boundaries & eco- biogeographic proxies",
                 "geographic boundaries & management units",
                 "management units",
                 "other", 
                 "other combinations")


### add them
indicators_full$defined_populations_nicenames <- factor(
    indicators_full$defined_populations_simplified,
    levels = levels(as.factor(indicators_full$defined_populations_simplified)),
    labels = nice_names)

# metadata
metadata$defined_populations_nicenames <- factor(
    metadata$defined_populations_simplified,
    levels = levels(as.factor(metadata$defined_populations_simplified)),
    labels = nice_names)

#check names match
select(metadata, defined_populations_nicenames, defined_populations_simplified)
levels(indicators_full$defined_populations_nicenames)
```



## Averaging multiassessments (alternative assessments)
Some taxa were assessed twice or more times, for example to account for uncertainty on how to divide populations. This information is stored in variable `multiassessment` of the metadata (created by `get_metadata()`). An example of taxa with multiple assessments:

```{r}
metadata %>%
filter(multiassessment=="multiassessment")  %>%
  select(taxonomic_group, taxon, country_assessment, multiassessment) %>%
  arrange(taxon, country_assessment) %>%
  head()
  
```

Alternative assessments allow to account for uncertainty in the number of populations or the size of them. We can examine how the indicators value species by species as done elsewhere in these analyses (see below "Values for indicator 1 and 2 for multiassessed species), but to examine global trends, some of the figures below use the average. **The averages are stored in a different column, labeled `indicator[1 or 2]_mean`.**

```{r}
indicators_averaged<-indicators_full %>%
  # group desired multiassessments
  group_by(country_assessment, multiassessment, taxon) %>%
  # estimate means
  mutate(indicator1_mean=mean(indicator1, na.rm=TRUE)) %>%
  mutate(indicator2_mean=mean(indicator2, na.rm=TRUE)) %>%
  # change NaN for NA (needed due to the NAs and 0s in the dataset)
  mutate_all(~ifelse(is.nan(.), NA, .)) 
```

Examples of how this looks to check it was done properly. For indicator 1:

A species assessed in different countries (Sweden and Belgium), with alternative assessment within Sweden. The average is computed only for the Swedish assessments
```{r}

indicators_averaged %>%
  filter(taxon == "Barbastella barbastellus") %>%
  select(taxon, country_assessment, multiassessment, indicator1, indicator1_mean)


```
Alternative assessments within a single country:
```{r}
indicators_averaged %>%
  filter(taxon == "Rana dalmatina") %>%
  select(taxon, country_assessment, multiassessment, indicator1, indicator1_mean, indicator2, indicator2_mean)

indicators_averaged %>%
  filter(taxon == "Notophthalmus perstriatus") %>%
  select(taxon, country_assessment, multiassessment,  indicator1, indicator1_mean, indicator2, indicator2_mean)

```

If one of the alternative assessments has NA, it is removed from the mean calculation, so that the average equals the value of the assessment that has data:
```{r}
indicators_averaged %>%
  filter(taxon == "Alouatta palliata mexicana") %>%
  select(taxon, country_assessment, multiassessment,  indicator1, indicator1_mean, indicator2, indicator2_mean)
```

A species can have alternative assessments for both indicators:
```{r}
indicators_averaged %>%
  filter(taxon == "Aphelocoma coerulescens") %>%
  select(taxon, country_assessment, multiassessment, indicator1, indicator1_mean, indicator2, indicator2_mean)

```


Because we will use the averages to show a single value for multiasssessed taxa, we can keep only the first record for multiassessed taxa.
```{r}
indicators_averaged_one<-indicators_averaged[!duplicated(cbind(indicators_averaged$taxon, indicators_averaged$country_assessment)), ]

```


Example of how this looks:
```{r}
indicators_averaged_one %>%
  filter(taxon == "Aphelocoma coerulescens") %>%
  select(taxon, country_assessment, multiassessment, indicator2, indicator2_mean, indicator1, indicator1_mean)


indicators_averaged_one %>%
  filter(taxon == "Rana dalmatina") %>%
  select(taxon, country_assessment, multiassessment, indicator2, indicator2_mean, indicator1, indicator1_mean)


indicators_averaged_one %>%
  filter(taxon == "Notophthalmus perstriatus") %>%
  select(taxon, country_assessment, multiassessment, indicator2, indicator2_mean, indicator1, indicator1_mean)

```

Total number of taxa (ie assessments averaging alternative assessments):
```{r}
nrow(indicators_averaged_one)
```


## General description of records and taxa assessed by country

Records by country, including taxa assessed more than once (see below for details on this)

```{r  taxa by country, out.width="700px", out.height="400px"}
ggplot(metadata, aes(x=country_assessment)) + 
  geom_bar(stat = "count") +
  xlab("") +
  ggtitle("Number of taxa assessed by country, including taxa assed more than once") +
  theme_light()

```

To explore what kind of taxa countries assessed regardless of if they assessed them once or more, we are going to use the subset `indicators_averaged_one`, were we averaged the indicators and kept only 1 record per assessment. 

How many taxa were assessed (i.e. counting only once taxa that were assessed multiple times)?
```{r}
# how many?
nrow(indicators_averaged_one)
```

Plot taxa assessed excluding duplicates, i.e. the real number of taxa assessed:

```{r, out.width="700px", out.height="400px"}
p1<-ggplot(indicators_averaged_one, aes(x=country_assessment)) + 
  geom_bar(stat = "count") +
  xlab("") +
  ggtitle("Number of taxa assessed by country") +
  theme_light()
p1

```

Of which countries and taxonomic groups are the taxa that were assessed more than once?
```{r multiassessed}
p2<- indicators_averaged_one %>% # we use the _unique dataset so that multiassesed records are counted only once
        filter(multiassessment=="multiassessment") %>%

ggplot(aes(x=taxonomic_group, fill=country_assessment)) + 
  geom_bar(stat = "count") +
  theme(axis.text.x = element_text(angle = 45)) +
  labs(fill="Country") +
  xlab("") +
  ggtitle("Number of taxa assessed more than once") +
  theme_light()

p2
```


Total number of taxon with alternative assessments:
```{r}
nrow(indicators_averaged_one %>% filter(multiassessment == "multiassessment"))
```

Number of alternative assessments:
```{r}
nrow(indicators_averaged%>% filter(multiassessment == "multiassessment"))
```


How many alternative assessments there are per multiassessed taxon?
```{r}
x<-indicators_averaged %>% select(country_assessment, multiassessment, taxon) %>%
                         filter(multiassessment == "multiassessment") %>%
                        group_by(taxon)  %>%
                        summarise(n=n())

kable(x)

mean(x$n)
sd(x$n)
median(x$n)
```

### Supplementary Figure S1: Number of species and multiassessed species per country

```{r}
plot_grid(p1 + ggtitle(""),
          p2 + ggtitle(""), ncol = 1, labels = c("A)", "B)"))

ggsave("FigS1.pdf", width = 20, height = 24  , units = "cm")

```


### Heatmap of the taxa assessed by country (counting multiassessments only once)

We aimed to represent different taxonomic groups within animals (amphibians, birds, fishes, invertebrates, mammals and reptiles), plants (angiosperms, bryophytes, gymnosperms and pteridophytes), fungi and others (e.g. lichens). Order levels to represent those categories:

```{r}
indicators_averaged_one$taxonomic_group<- factor(indicators_averaged_one$taxonomic_group,
                                                 levels = c("amphibian", "bird", "fish", "invertebrate", "mammal", "reptile", "angiosperm", "bryophyte", "gymnosperm", "pteridophytes", "fungus", "other"))
```

Make a heatmap

```{r}

## Agregate data to get counts 

agg_data <- indicators_averaged_one %>% # we use the _unique dataset so that multiassesed records are counted only once
        filter(multiassessment!="multiassessment") %>%
        group_by(country_assessment, taxonomic_group) %>%
        summarize(count = n())

# country names in desired order
agg_data$country_assessment <- factor(agg_data$country_assessment, 
                                      levels = rev(levels(agg_data$country_assessment)))
  
  
## Create a heat map
p_heat<- ggplot(agg_data, aes(x = taxonomic_group, y = country_assessment, fill = count)) + 
  geom_tile() + 
  scale_fill_gradient(low = "lightblue", high = "darkblue") +  # Adjust color scale as needed
  labs(x = "",
    y = "",
    fill = "Number of taxa"
  ) + 
  scale_x_discrete(position = "top") +
  theme_light() + 
  theme(panel.border = element_blank(), axis.text.x = element_text(angle = 90, hjust = 0),
    legend.position = "right", text = element_text(size = 13),
    panel.grid.major = element_blank(), panel.grid.minor = element_blank(), # remove background
  )
p_heat
  
```




## Population size data (Has Nc or Ne? what type of Nc?)

Transform NAs to insuff_data_species, since effectively they are the same:

```{r}
# check NAs
summary(as.factor(metadata$popsize_data))


# Replace NA to insuff_data_species
metadata <-metadata %>% 
           mutate(popsize_data=replace_na(popsize_data, "insuff_data_species"))

# check
summary(as.factor(metadata$popsize_data))
```

Re-order  popsize_data to have inssuficient data at the end in the plots
```{r}
metadata$popsize_data<-factor(metadata$popsize_data, 
                              levels= c("insuff_data_species", "data_for_species","yes"))

```



### Supplementary Figure S5: Population size data availability by country

Countries have population size data (Nc or Ne) regardless of the taxonomic group. The last panel includes the entire dataset:

```{r}
## Duplicate data with an additional column "facet"

df<-CreateAllFacet(metadata, "country_assessment")

# order with "all" as last
df$facet <- factor(df$facet, levels=c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA", "all"))

# Plot
ggplot(df, aes(x=taxonomic_group, fill=popsize_data)) + 
  geom_bar(stat = "count", color="white") +
  coord_flip() +
  facet_wrap(~facet, ncol = 5, scales="free_x") +
  scale_fill_manual(values=c("#2ca02c", "#1f77b4", "grey80"),
                    breaks=c("yes", "data_for_species", "insuff_data_species"),
                    labels=c("Population level", "Species or subspecies level", "Insufficient data")) +  
  labs(fill="Population size data availability",
       x="",
       y="Number of taxa (including records of taxa assessed more than once)") +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="top")

ggsave("FigS5.pdf", width = 31, height = 25  , units = "cm")

```

Population size data availability in the entire dataset, by taxon group:
```{r}
ggplot(metadata, aes(x=taxonomic_group, fill=popsize_data)) + 
                geom_bar(stat = "count", color="white") +
                coord_flip() +
  scale_fill_manual(values=c("#2ca02c", "#1f77b4", "grey80"),
                    breaks=c("yes", "data_for_species", "insuff_data_species"),
                    labels=c("Population level", "Species or subspecies level", "Insufficient data")) +  
                labs(fill="Population size data availability",
                     x="",
                     y="Number of taxa") +
                theme_light() +
                theme(legend.position="right")

```

Population size data availability in the entire dataset, by taxon country:
```{r}
p.popsize<-ggplot(metadata, aes(x=country_assessment, fill=popsize_data)) + 
                geom_bar(stat = "count", color="white") +
                coord_flip() + 
                scale_x_discrete(limits=rev) +
   scale_fill_manual(values=c("#2ca02c", "#1f77b4", "grey80"),
                    breaks=c("yes", "data_for_species", "insuff_data_species"),
                    labels=c("Population level", "Species or subspecies level", "Insufficient data")) +  
   labs(fill="Population size data availability",
                     x="",
                     y="Number of taxa") +
                theme_light() +
                theme(text = element_text(size = 13), panel.border = element_blank(), legend.position="right")
p.popsize
```

Species level yes/no table with percentages
```{r}
#total n
nrow(indicators_full)

# table
df<- indicators_full %>%
     group_by(popsize_data) %>%
   summarise(n=n(),
             percentage = (n / nrow(metadata)) * 100)
   
kable(df, digits = 0)

```

### Ne data yes or not? & Type of Nc data 

Ne available by taxa? (species level)
```{r}
p1<- metadata %>% 
  filter(!is.na(ne_pops_exists)) %>% 
  filter(ne_pops_exists!="other_genetic_info") %>%
    ggplot(aes(x=country_assessment, fill=ne_pops_exists)) + 
  geom_bar(color="white") +
scale_fill_manual(labels=c("No", "Yes"),
                      breaks=c("no_genetic_data", "ne_available"),
                      values=c("#ff7f0e", "#2ca02c")) +
xlab("") +
ylab("Number of taxa") +
labs(fill="Ne available \n(from genetic data)")  +
theme_light() +
theme(text = element_text(size = 13), legend.position = "right", panel.border = element_blank())

p1

```

Nc data available by taxa? (species level)
```{r}
p2<-metadata %>%
  filter(!is.na(nc_pops_exists)) %>%
    ggplot(aes(x=country_assessment, fill=nc_pops_exists)) +
    geom_bar(color="white") +
scale_fill_manual(labels=c("No", "Yes"),
                      breaks=c("no", "yes"),
                      values=c("#ff7f0e", "#2ca02c")) +
    labs(fill="Nc available") +
    xlab("") +
    ylab("Number of taxa") +
    theme_light() +
    theme(text = element_text(size = 13), legend.position = "right", panel.border = element_blank())
p2
```


What kind of Nc data? (dodge bars) This is at population level. 
```{r}
ind1_data %>%
  filter(!is.na(NcType)) %>%
  ggplot(aes(x=country_assessment, fill=NcType))+
  geom_bar(position = "dodge", color="white") +
  scale_fill_manual(labels=c("Point", "Range \nor qualitative", "Unknown"),
                      breaks=c("Nc_point", "Nc_range", "unknown"),
                      values=c("#0072B2", "#E69F00", "grey80")) +
  xlab("") +
  ylab("Number of populations") +
  labs(fill="Type of Nc data \nby population") +
  theme_light() +
  theme(text = element_text(size = 13), legend.position = "right", panel.border = element_blank())

 
```

What kind of Nc data? (fill bars). This is at population level. 

```{r}
p3<-ind1_data %>%
  filter(!is.na(NcType)) %>%
  ggplot(aes(x=country_assessment, fill=NcType))+
  geom_bar(position = "fill", color="white") +
  scale_fill_manual(labels=c("Point", "Range \nor qualitative", "Unknown"),
                      breaks=c("Nc_point", "Nc_range", "unknown"),
                      values=c("#0072B2", "#E69F00", "grey80")) +
  xlab("") +
  ylab("Proportion of populations") +
  labs(fill="Type of Nc data \nby population") +
  theme_light() +
  theme(text = element_text(size = 13), legend.position = "right", panel.border = element_blank())
p3
 
```

Data availability at the population level cosidering Ne and Nc combined. This plot shows where data came from for the Ne value used for estimating the indicator.

```{r}



p4<-ind1_data %>%
  # NA as a category
  mutate(Ne_calculated_from = replace_na(Ne_calculated_from, "NA")) %>%
  # reorder to have NA at the end (here at the start because we will use coord_flip below)
  mutate(Ne_calculated_from = factor(Ne_calculated_from, levels=c("NA", "NcRange ratio", "NcPoint ratio", "genetic data")))  %>%
  ggplot(aes(x=country_assessment, fill=Ne_calculated_from))+
  geom_bar(position = "fill", color="white") +
  scale_fill_manual(labels=c("Genetic data", "Nc ratio (point)", "Nc ratio (range or qualitative)", "Missing data"),
                      breaks=c("genetic data", "NcPoint ratio", "NcRange ratio", "NA"),
                      values=c("darkgreen", "#3388CC", "#E69F00", "grey80")) +
  xlab("") +
  scale_x_discrete(limits=rev) + 
  ylab("Percentage of populations") +
  scale_y_continuous(labels = scales::percent) + # show axis in % instead of decimal
  labs(fill="Data used to estimate Ne") +
  theme_light() +
  coord_flip() +
  theme(text = element_text(size = 13), legend.position = "bottom", panel.border = element_blank())
p4

```

### Supplementary Figure S4: Ne and Nc data availabiltiy by taxa
```{r}
plot_grid(p1 + theme(legend.justification = c(0,.5)),  # legend.justification is used to aling legends
          p2 + theme(legend.justification = c(0,.5)),
          ncol=1, align = "v", labels=c("A)", "B)"), vjust = .95)  

ggsave("FigS4.pdf", width = 23, height = 25  , units = "cm")

```


## Range of values for Ne and Nc data

Range of Ne values by taxonomic group, without possible outliers (Ne > 100000)

```{r}
ind1_data %>%
  filter(Ne < 100000) %>%
  filter(!is.na(Ne))  %>%
  
  ggplot(aes(x=taxonomic_group, y=Ne)) +
  geom_boxplot(color="grey50") +
  geom_jitter(size=.5, width = 0.1, color="darkred") +
  xlab("") +
  theme_light() +
  theme(axis.text.x = element_text(angle = 45))
```

Check outliers
```{r}
ind1_data %>% 
  filter(Ne > 100000) %>%
  select(country_assessment, name_assessor, taxon, taxonomic_group, Ne, NeLower, NeUpper, multiassessment, population)
```


Range of Nc values (actual data point provided) by taxonomic group. Without possible outliers.

```{r}
ind1_data %>%
  filter(!is.na(NcPoint))  %>%
  filter(NcPoint < 10000000) %>%
 
  ggplot(aes(x=taxonomic_group, y=NcPoint)) +
  geom_boxplot(color="grey50") +
  geom_jitter(size=.5, width = 0.1, color="darkred") +
  xlab("") +
  theme_light() +
  theme(axis.text.x = element_text(angle = 45))
```

Check outliers
```{r}
ind1_data %>% 
  filter(NcPoint > 10000000) %>% 
  select(country_assessment, name_assessor, taxon, taxonomic_group, population, NcPoint, NcLower, NcUpper, multiassessment, population)
```

Range of Ne values by taxonomic group from different sources. Without possible outliers.

```{r}
ind1_data %>%
  filter(!is.na(Ne_combined))  %>%
  filter(Ne < 100000) %>%
 
  ggplot(aes(x=taxonomic_group, y=Ne_combined)) +
  geom_boxplot(color="grey50") +
  geom_jitter(size=.5, width = 0.1, color="darkred") +
  xlab("") +
  theme_light() +
  theme(axis.text.x = element_text(angle = 45))
```

Range of Ne values by taxonomic group from different sources. Zoom to Ne < 10,000

```{r}
ind1_data %>%
  filter(!is.na(Ne_combined))  %>%
  filter(Ne < 10000) %>%
 
  ggplot(aes(x=taxonomic_group, y=Ne_combined)) +
  geom_boxplot(color="grey50") +
  geom_jitter(size=.5, width = 0.1, color="darkred") +
  xlab("") +
  theme_light() +
  theme(axis.text.x = element_text(angle = 45))
```

## Number of populations with Ne < 50 from genetic or NcPoint data

The more worrying threshold, where the effects of inbreeding become more pronounced in the short-term (more immediate risk of extinction), occurs at Ne < 50. We cannot estimate how many populations are below this threshold from all our data, since the NcRange variable includes qualitative range data (eg. "less 5000 by much") that we can translate into more/less than Ne 500, but not Ne 50. 

Therefore to count how many populations are Ne < 50 we first selected the populations with Ne or NcPoint data:

```{r}
forNe50<- ind1_data %>%
          # keep only rows without missing data in Ne or NcPoint
          filter(!is.na(Ne) | !is.na(NcPoint)) %>%
          # keep more relevant variables:
          select(country_assessment, taxon, Ne, MethodNe, NcPoint, NcRange, Ne_from_Nc, Ne_calculated_from, Ne_combined)

forNe50

```

Total number of populations with Ne or Nc_point data:
```{r}
nrow(forNe50)
```


We then can count how many of them are below Ne 50:
```{r}
# select pops Ne_combined (coming from either Ne genetic or NePoint converted with the ratio)
Neless50<- forNe50  %>% filter(Ne_combined < 50)

# how many?
nrow(Neless50)

# % of the the total number of pops with Ne or Nc_point data:
round(nrow(Neless50) / nrow(forNe50) * 100)
```

Check by origin of the Nc data
```{r}
Neless50 %>% ggplot(aes(x=Ne_combined, fill=Ne_calculated_from)) +
  geom_histogram(alpha=0.6, position = 'identity') + theme_light()
  
```


## Missing data on extant and extinct populations

We have NA in Proportion of maintained populations (indicator) because in some cases the number of extinct populations is unknown, therefore the operation cannot be computed. 

### Counts
Total records with NA in extant populations:
```{r}
sum(is.na(indicators_full$n_extant_populations))
```

Taxa with NA in extant populations:
```{r}
indicators_full %>%
  filter(is.na(n_extant_populations)) %>%
    select(country_assessment, taxonomic_group, taxon, n_extant_populations, n_extint_populations)
```

Total taxa with NA in **extinct** populations:
```{r}
sum(is.na(indicators_full$n_extint_populations))
```

Do taxa with NA for extant also have NA for extinct?

```{r}
indicators_full$taxon[is.na(indicators_full$n_extant_populations)] %in% indicators_full$taxon[is.na(indicators_full$n_extint_populations)]
```

So out of the `r nrow(indicators_full)`, we have **`r sum(is.na(indicators_full$n_extint_populations))` records with NA in n_extinct** and **`r sum(is.na(indicators_full$n_extant_populations))` records with NA in n_extant**. Of them, `r sum(indicators_full$taxon[is.na(indicators_full$n_extant_populations)] %in% indicators_full$taxon[is.na(indicators_full$n_extint_populations)])` have NA in both n_extant and n_extinct.


### Plot missing data extinct populations
```{r}

p5<-indicators_full %>%
  ggplot(aes(x=country_assessment, fill=!is.na(n_extint_populations))) + # Notice this shows which taxa DO NOT have NA, ie exctint pops are KNOWN
  geom_bar(color="white") +
  scale_fill_manual(labels=c("Number of populations known", "Number of populations unknown"),
                     breaks =c("TRUE", "FALSE"),
                     values=c("#2ca02c", "#ff7f0e")) + 
  labs(fill="Data availability on extinct \npopulations") +
  xlab("") + ylab("Number of taxa") +
  theme_light() +
  theme(text = element_text(size = 13), legend.position = "right", panel.border = element_blank())
p5
```


### Supplementary Figure S3: Missing data in extinct populations by country and method and by taxonomic group

Missing data in number of extinct populations by method to define populations:

```{r, out.width="850px", out.height="650px"}
pa<-indicators_full %>%
  ggplot(aes(x=defined_populations_nicenames, fill=!is.na(n_extint_populations))) +
  geom_bar(color="white") +
  coord_flip()+ 
  scale_fill_manual(labels=c("Number of populations \nknown", "Number of populations \nunknown"),
                     breaks =c("TRUE", "FALSE"),
                     values=c("#2ca02c", "#ff7f0e")) + 
  labs(fill="Data availability on \nextinct populations") +
  xlab("") + ylab("Number of taxa") +
  facet_wrap(country_assessment ~., nrow = 3, scales="free_x") +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="top")
 pa 
```

Missing data in number of extinct populations by method to define populations:

```{r, out.width="850px", out.height="650px"}
pb<-indicators_full %>%
  ggplot(aes(x=taxonomic_group, fill=!is.na(n_extint_populations)))+
  geom_bar(color="white") +
  coord_flip()+ 
  scale_fill_manual(labels=c("Number of populations \nknown", "Number of populations \nunknown"),
                     breaks =c("TRUE", "FALSE"),
                     values=c("#2ca02c", "#ff7f0e")) + 
  labs(fill="Data availability on \nextinct populations") +
  xlab("") + ylab("Number of taxa") +
  facet_wrap(country_assessment ~., nrow = 3, scales="free_x") +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none")
pb  
```

Single figure:
```{r}
plot_grid(pa, pb, 
          align = "v", ncol=1,
          rel_heights = c(1.1, 1),
          labels=c("A)", "B)"))

ggsave("FigS3.pdf", width = 31, height = 50  , units = "cm")
```




## Main Figure 1: Taxa by country and data availability to estimate Ne indciator (origion of data to estimate Ne) and PM indicator (missing data on pop extinction):

Distribution of Nc, Ne and types of Ne in a single figure with 4 panels, using count for a, b & c, and proportions for d:


```{r, out.width="900px", out.height="1250px"}
# plot

plot_grid(p_heat + theme(legend.position = "right", legend.justification = c(0,.5)), # legend.justification aligns legends
         
           p5 + coord_flip() + 
            scale_x_discrete(limits=rev) +
            theme(legend.position = "right", legend.justification = c(0,.5)),
          
          p.popsize + theme(legend.position = "right", legend.justification = c(0,.5)),
          
          p4  + theme(legend.position = "right", legend.justification = c(0,.5)), 
          
          ncol = 1, labels = c("A)", "B)", "C)", "D)"), align = "v",
          rel_heights = c(1.3, 1, 1,1))


ggsave("Fig1.pdf", width = 17, height = 33  , units = "cm")

```


## Main Figure 2: Method to define populations used by country and taxa (alluvial)

Reformat data
```{r}

# reformat data
foralluvial<-metadata %>% group_by(country_assessment, defined_populations_nicenames, taxonomic_group) %>%
             summarise(n=n()) 

# define colors
my_cols<- simplifiedmethods_colors

# we need a vector of colors by country for each row of the dataset, so:
methodspop<-as.factor(foralluvial$defined_populations_nicenames)
levels(methodspop)<-my_cols
methodspop<-as.vector(methodspop)
head(methodspop)
```

Plot
```{r}
alluvial(foralluvial[,1:3], freq = foralluvial$n,
         col=methodspop, 
         blocks=FALSE,
         gap.width = 0.5,
         cex=.8, 
         xw = 0.1,
         cw = 0.2,
         border = NA,
         alpha = .7)


pdf("Fig2.pdf", width = 9, height = 7)
# plot
alluvial(foralluvial[,1:3], freq = foralluvial$n,
         col=methodspop, 
         blocks=FALSE,
         gap.width = 0.5,
         cex=.8, 
         xw = 0.1,
         cw = 0.2,
         border = NA,
         alpha = .7)
dev.off()

```

Get count to add numbers outside R:
```{r}
# n for country
metadata %>% group_by(country_assessment) %>%
             summarise(n=n()) 

# n for taxonomic group
metadata %>% group_by(taxonomic_group) %>%
             summarise(n=n()) 
```


## Exploratory plots for the association of distribution range (restricted vs wide) on the indicators

All the following plots and analyses consider the average of multiassessed species (variable `_mean`), so that they are shown only once.

To have nicer looking plots, change "wide_ranging" for "wide ranging":
```{r}
indicators_averaged_one$species_range<-gsub("wide_ranging", "wide ranging", indicators_averaged_one$species_range)
```


### Indicator 1 (Ne>500)

Plot Indicator 1 by type of range in the entire dataset. Filtering NA in species range:

```{r indicator1 by range}
# get sample size by desired category
sample_size <- indicators_averaged_one  %>%
                    filter(!is.na(indicator1_mean)) %>% 
                    filter(!is.na(species_range)) %>% 
                    group_by(species_range) %>% summarize(num=n())

# plot
p1<-indicators_averaged_one %>% 
    filter(!is.na(indicator1_mean)) %>% 
    filter(!is.na(species_range)) %>% 
  # add sampling size 
  left_join(sample_size) %>% 
  mutate(myaxis = paste0(species_range, " (n= ", num, ")")) %>%

  # plot
  ggplot(aes(x=myaxis, y=indicator1_mean , fill=species_range)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
      scale_fill_manual(breaks=c("wide ranging", "restricted", "unknown"),
                       labels=c("wide ranging", "restricted", "unknown"),
                       values=c("#00BFC4", "#F8766D", "grey80")) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", text= element_text(size=20))
p1
```

###  Plot Ne Indicator by country and type of range. Remove "unknown" and NA for better visualization.

```{r indicator1 by country and range}
### Duplicate dataframe to have a column with "all data" for faceting
df<-CreateAllFacet(indicators_averaged_one, "country_assessment")

# order with "all" as last
df$facet <- factor(df$facet, levels=c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA", "all"))

## plot
df  %>% 
  # filter out "unknown" range
  filter(species_range !="unknown") %>% 
  
# plot
ggplot(aes(x=species_range, y=indicator1_mean , fill=species_range)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
      scale_x_discrete(breaks=c("wide ranging", "restricted"),
                        labels=c("wide ranging", "restricted")) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", text= element_text(size=15)) +
      facet_wrap(~facet, ncol = 5) +
      theme(panel.spacing = unit(1.5, "lines"))  
  
```



### Indicator 2 (mantained populations)

Plot Indicator 2 by type of range in the entire dataset. Filtering NA in species range:

```{r indicator2 by range}
# get sample size by desired category
sample_size <- indicators_averaged_one  %>%
                    filter(!is.na(indicator2_mean)) %>% 
                    filter(!is.na(species_range)) %>% 
                    group_by(species_range) %>% summarize(num=n())

# plot
p2<-indicators_averaged_one %>% 
    filter(!is.na(indicator2_mean)) %>% 
    filter(!is.na(species_range)) %>% 
  # add sampling size 
  left_join(sample_size) %>% 
  mutate(myaxis = paste0(species_range, " (n= ", num, ")")) %>%

  # plot
  ggplot(aes(x=myaxis, y=indicator2_mean , fill=species_range)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
      coord_flip() +
      scale_fill_manual(breaks=c("wide ranging", "restricted", "unknown"),
                       labels=c("wide ranging", "restricted", "unknown"),
                       values=c("#00BFC4", "#F8766D", "grey80")) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", text= element_text(size=20))
p2
```

Plot Indicator 2 by country and type of range. We remove NA and unknown for better visualization.

```{r indicator2 by country and range}
### Duplicate dataframe to have a column with "all data" for faceting
df<-CreateAllFacet(indicators_averaged_one, "country_assessment")

# order with "all" as last
df$facet <- factor(df$facet, levels=c("Australia", "Belgium", "Colombia", "France", "Japan", "Mexico", "South Africa", "Sweden", "USA", "all"))

## plot
df  %>% 
  # filter out "unknown" range
  filter(species_range !="unknown") %>% 
  
# plot
ggplot(aes(x=species_range, y=indicator2_mean , fill=species_range)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
      coord_flip() +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", text= element_text(size=15)) +
      facet_wrap(~facet, ncol = 5) +
      theme(panel.spacing = unit(1.5, "lines"))  
  
```



### Single plot PM and Ne indicators by range type
```{r,  out.height="900px", out.width="800px"}
plot_grid(p1, p2,  ncol=1, align = "v", labels=c("a)", "b)"))  
```


## Sampling size of different variables useful to interpret the statistical models run below

The plots and tables below are meant to visualize the sampling size and data distribution of some of the variables used in the models below. Data is a subset  filtering outliers (>500 populations) and using the simplified methods (see above). Multiassessed species are considered independently (each assessment is a data point).

### Number of maintained populations by country and method

Number of maintained populations by country and method is useful to interpret the models that would be run below.
```{r}
indicators_full %>% 
  filter(n_extant_populations<500) %>% # filter outliers
  # order countries vertically by similar number of pops
  mutate(country_assessment = factor(country_assessment, 
                                     levels=c("Colombia", "Australia", "Belgium",
                                               "Mexico", "France", "USA", 
                                               "South Africa", "Japan", "Sweden"))) %>%
  ggplot(aes(x=defined_populations_nicenames, y=n_extant_populations, 
             fill=defined_populations_nicenames, color=defined_populations_nicenames)) +
          geom_boxplot() +
          geom_jitter(size=.3, width = 0.1, color="black") +
  coord_flip() +
  facet_wrap(country_assessment ~ ., nrow=3, scales="free_x") +
  xlab("")  +
  ylab("Number of maintained populations") +
  scale_fill_manual(values=alpha(simplifiedmethods_colors, .3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_x_discrete(limits=rev) +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        text = element_text(size = 15)) 


```

### Ne values 

```{r}
ind1_data %>% 
  filter(Ne_combined < 100000) %>% # filter outliers

  ggplot(aes(x=defined_populations_simplified, y=Ne_combined, 
             color=Ne_calculated_from)) +
          geom_boxplot(position = "dodge") +
          geom_jitter(position = position_dodge(width = 0.75)) +  
          facet_wrap(country_assessment ~ ., nrow=3) +
          coord_flip() +
          theme_light()

```

Zoom to Ne 500
```{r}
ind1_data %>% 
  filter(Ne_combined < 100000) %>% # filter outliers

  ggplot(aes(x=defined_populations_simplified, y=Ne_combined, 
             color=Ne_calculated_from)) +
          ylim(0,2000)+
          geom_boxplot(position = "dodge") +
          geom_jitter(position = position_dodge(width = 0.75)) +  
          facet_wrap(country_assessment ~ ., nrow=3) +
          coord_flip() +
          theme_light()

```


Summary table for sampling size by method and source of Ne:
```{r}
x<- ind1_data %>% 
  filter(!is.na(Ne_calculated_from)) %>% 
  group_by(defined_populations_simplified, Ne_calculated_from) %>%
                summarise(n=n())
kable(x)
```

Same as above but adding country:
```{r}
x<- ind1_data %>% 
  filter(!is.na(Ne_calculated_from)) %>% 
  group_by(country_assessment, defined_populations_simplified, Ne_calculated_from) %>%
                summarise(n=n())
kable(x)
```


### Distribution type (wide / restricted)

```{r}
x<-indicators_full %>% filter(species_range !="unknown") %>%
                    group_by(defined_populations_nicenames, species_range) %>% 
                    summarise(n=n())

kable(x)
```



### Supplementary Figure S6: number of mantained populations by method to define populations and range type

Facet by range type:
```{r}
p<-indicators_full %>% 
  filter(!is.na(n_extant_populations)) %>% 
  filter(n_extant_populations<500) %>%
  filter(species_range !="unknown") %>%
  filter(!is.na(species_range)) %>%
  
  ggplot(aes(x=defined_populations_nicenames, y=n_extant_populations)) +
          geom_boxplot(aes(color=defined_populations_nicenames,
                           fill=defined_populations_nicenames)) + 
    xlab("") + ylab("Number of maintained populations") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0, 0, 0, 0), "cm")) + # this is used to decrease the space between plots
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  facet_wrap("species_range", ncol=1) + 
  geom_jitter(size=.4, width = 0.1, color="black") +
  scale_x_discrete(limits=rev)

p

ggsave("FigS6.pdf", width = 18, height = 19  , units = "cm")
```



## Statistical models: test for associations between method used to define populations / range type on the number of populations and the indicator values

The analyses and plots below us a subset of data filtering outliers (>500 populations) and using the simplified methods (see above). Multiassessed species are considered independently (each assessment is a data point).

### (a) Does the number of maintained pops vary with method used? 

First we tested whether the different methods reported in this study were associated with varying numbers of populations obtained. For this analysis, we also controlled for range type, as we expect species with wider ranges to plausibly have more populations than species with narrower ranges.

Plot number of populations by method.
```{r, out.height="500px", out.width="1064px"}
# Prepare data for plot with nice labels:
# sample size of TOTAL populations
sample_size <- indicators_full %>%
                    filter(!is.na(n_extant_populations)) %>% 
                    filter(n_extant_populations<500) %>%
                    group_by(defined_populations_nicenames) %>% summarize(num=n())

# custom axis
## new dataframe
df<-indicators_full %>% 
  filter(!is.na(n_extant_populations)) %>% 
  filter(n_extant_populations<500) %>%
    # add sampling size 
  left_join(sample_size) %>%
  mutate(myaxis = paste0(defined_populations_nicenames, " (n= ", num, ")")) %>%
#myaxis needs levels in the same order than defined_populations_nicenames
  mutate(myaxis = factor(myaxis, 
                  levels=levels(as.factor(myaxis))[c(1,12,2:11,13)])) # reorders levels


# plot for number of pops
  pa<- df %>%
  ggplot(aes(x=myaxis, y=n_extant_populations, color=defined_populations_nicenames,
                                               fill=defined_populations_nicenames)) +
          geom_boxplot() + xlab("") + ylab("Number of maintained populations") +
          geom_jitter(size=.4, width = 0.1, color="black") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0, 0, 0, 0), "cm")) + # this is used to decrease the space between plots
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_x_discrete(limits=rev) +  
  theme(text = element_text(size = 13))
pa

```

Prepare data for model (remove outliers, "unknown" category and NA in desired variable) and check n:
```{r}
# remove missing data 
data_for_model<-indicators_full %>% 
                      filter(!is.na(n_extant_populations)) %>%
                      filter(species_range !="unknown") %>% # we remove "unknonw" because its n is too low, thus unbalancing the model
                      filter(n_extant_populations<500) # doesn't make a difference in the test below, but useful for plots

# check n per method
table(data_for_model$defined_populations_simplified)

# total n
nrow(data_for_model)

# re-level to use geographic boundaries as reference category for the analysis
data_for_model$defined_populations_simplified<-relevel(as.factor(data_for_model$defined_populations_simplified),
                                                       ref="geographic_boundaries")

# make sure specis range is a factor
data_for_model$species_range<-as.factor(data_for_model$species_range)

```

Run model asking: Does the number of maintained pops vary with method and range?

```{r}
m.a1<-glmer(data_for_model$n_extant_populations ~ data_for_model$defined_populations_simplified + data_for_model$species_range + (1|data_for_model$country_assessment), family ="poisson")

summary(m.a1)
```

Considering the role of method was so important for determining the number of populations, we also tested whether this effect remained after removing “wide-ranging” from the model. The objective here was to test whether method alone would also produce varying numbers of populations, for example if species rangedness were unknown.

Does the number of maintained pops vary with method used? (does method still influence number of populations if we exclude range type from the model):

```{r}
m.a2<-glmer(data_for_model$n_extant_populations ~ data_for_model$defined_populations_simplified + 
            (1|data_for_model$country_assessment), family ="poisson")
```

See results:
```{r}
summary(m.a2)
```

Extending from this result, we also tested whether species range alone is an important predictor of the number of extant populations, as species range is determined by the geographic spread of the species, but not necessarily fragmentation

Does the number of maintained pops vary with range?
```{r}
m.a3<-glmer(n_extant_populations ~ species_range + (1|country_assessment), family = "poisson", data = data_for_model)

summary(m.a3)
```



### (b) Does the proportion of maintained populations (indicator2) vary with method used to define populations?

Our next goal was to determine whether study design (i.e. clustering method to define populations) and/or species-level variables (number of populations, range type) appropriately were associated with the measurement of the genetic indicators. 


Plot PM indicator by method to define populations:

```{r}
# Prepare data for plot with nice labels:
# sample size of TOTAL populations
sample_size <- indicators_full %>%
                    filter(!is.na(indicator2)) %>% 
                    filter(n_extant_populations<500) %>% 
                    group_by(defined_populations_nicenames) %>% summarize(num=n())

# custom axis
## new dataframe
df<-indicators_full %>% 
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator2)) %>% 
    # add sampling size 
  left_join(sample_size) %>%
  mutate(myaxis = paste0(defined_populations_nicenames, " (n= ", num, ")")) %>%
#myaxis needs levels in the same order than defined_populations_nicenames
  mutate(myaxis = factor(myaxis, 
                  levels=levels(as.factor(myaxis))[c(1,12,2:11,13)])) # reorders levels



## plot for Proportion of maintained populations (indicator)
pb<- df %>%
  filter(n_extant_populations<500) %>%
  ggplot(aes(x=myaxis, y=indicator2, color=defined_populations_nicenames,    
                                     fill=defined_populations_nicenames)) +
          geom_boxplot() + xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
          geom_jitter(size=.4, width = 0.1, color="black") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0, 0, 0, 0), "cm")) + # this is used to decrease the space between plots) 
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_x_discrete(limits=rev) +
  theme(text = element_text(size = 13))
pb

```

Plot Scatter plot of indicator2 vs extant pops
```{r}
psupA<- indicators_full %>%
  # filter outliers with too many pops and missing data
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator2)) %>%
  filter(!is.na(n_extant_populations)) %>%
  filter(species_range !="unknown") %>%
  
  # plot
    ggplot(aes(x=n_extant_populations, y=indicator2, color=defined_populations_nicenames)) +
    geom_point() +
    theme_light() +
    scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
    theme(legend.position = "none") +
    ylab("Proportion of populations maintained within species \n(PM indicator)") +
    xlab("Number of maintained populations") +
    theme(text = element_text(size = 13))
psupA

psupA.1<- indicators_full %>%
  # filter outliers with too many pops and missing data
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator2)) %>%
  filter(!is.na(n_extant_populations)) %>%
  filter(species_range !="unknown") %>%
  
  # plot
    ggplot(aes(x=n_extant_populations, y=indicator2, color=species_range)) +
    geom_point() +
    theme_light() +
    theme(legend.position = "none") +
    ylab("Proportion of populations maintained within species \n(PM indicator)") +
    xlab("Number of maintained populations") +
    theme(text = element_text(size = 13))
psupA.1

```

First we want to test if the Proportion of maintained populations (indicator 2) vary with method used.

Prepare data for model (remove outliers and NA in desired variable) and check n:
```{r}
# remove missing data 
data_for_model<-indicators_full %>% 
                      filter(!is.na(indicator2)) %>%
                      filter(n_extant_populations<500) # doesn't make a difference in the test below, but useful for plots

# check n per method
table(data_for_model$defined_populations_simplified)

# total n
nrow(data_for_model)

# re-level to use geographic boundaries as reference category for the analysis
data_for_model$defined_populations_simplified<-relevel(as.factor(data_for_model$defined_populations_simplified),
                                                       ref="geographic_boundaries")

```

Run model asking: Does Proportion of maintained populations (indicator 2) vary with method used? Controlling for variation in indicator2 among countries:


```{r}
m.b1<-glmmTMB(indicator2 ~ defined_populations_simplified + (1|country_assessment), family = "ordbeta", data = data_for_model)
```

See results:
```{r}
summary(m.b1)
```

Given the preceding relationships detected between method, number of populations, and species’ range, we investigated associations between these variables and our indicator values in more detail, to aid in understanding the underlying mechanisms that were driving the association between method (especially genetic clusters) and indicator 2. That is, we hypothesised that the relationship between method and indicator 2 may be an indirect result of the association between method and number of populations and species range. 

First we added number of populations to our model testing the relationship between method and indicator 2

```{r}
m.b2<-glmmTMB(indicator2 ~ defined_populations_simplified + n_extant_populations + (1|country_assessment), family = "ordbeta", data = data_for_model)

summary(m.b2)
```

Then we tested (see plot psupA) if there is a relationship between number of maintained populations and the PM indicator, overall, and/or with some methods?

Prepare data for model (remove outliers and NA in desired variable) and check n:
```{r}
# remove missing data 
data_for_model<-indicators_full %>% 
                      filter(!is.na(indicator2)) %>%
                      filter(!is.na(n_extant_populations)) %>%
                      filter(n_extant_populations<500) # doesn't make a difference in the test below, but useful for plots

# check number of methods
length(unique(data_for_model$defined_populations_simplified))

# check n per method
table(data_for_model$defined_populations_simplified)

# total n
nrow(data_for_model)

# re-level to use geographic boundaries as reference category for the analysis
data_for_model$defined_populations_simplified<-relevel(as.factor(data_for_model$defined_populations_simplified),
                                                       ref="geographic_boundaries")

```

We tested for a relationship between number of populations alone with indicator 2 in our dataset (i.e. when not controlling for method). 

Does number of populations alone affect indicator2 (i.e. not controlling for method)?:

```{r}
msupA1 <- glmmTMB(indicator2 ~ n_extant_populations + (1|country_assessment), family = "ordbeta", data= data_for_model)

```

Summary:
```{r}
summary(msupA1)
```

But, there were statistically significant interactions between number of populations and some of the methods used, on indicator 2. 

Does the effect of method on indicator2 depend on number of maintained pops?

```{r}
# run model
msupA2 <- glmmTMB(indicator2 ~ defined_populations_simplified + n_extant_populations + defined_populations_simplified*n_extant_populations + (1|country_assessment), family = "ordbeta", data = data_for_model)

```

Summary:

```{r}
summary(msupA2)
```

Because the method used to define a population appears to be important for these relationships, we conducted an additional analysis to simplify our analysis to only those species for which a single method was used to determine population clusters, and repeated the model presented above (evaluating a possible interaction between method and number of populations on indicator 2).

First, subset the data to only those taxa where a single method was used: 

```{r}
ind2_single_methods<-indicators_full %>% 
                      filter(!is.na(indicator2)) %>% 
                      filter(n_extant_populations<500) %>%  # doesn't make a difference in the test below, but useful for 
                      filter(defined_populations_simplified=="genetic_clusters" | 
                             defined_populations_simplified=="geographic_boundaries" |
                             defined_populations_simplified=="eco_biogeo_proxies" | 
                             defined_populations_simplified=="management_units" |
                             defined_populations_simplified=="dispersal_buffer")


# check number of methods
length(unique(ind2_single_methods$defined_populations_simplified))

# check n by method
table(ind2_single_methods$defined_populations_simplified)

# check n total
nrow(ind2_single_methods)

# re-level to use geographic boundaries as reference category for the analysis
ind2_single_methods$defined_populations_simplified<-relevel(as.factor(ind2_single_methods$defined_populations_simplified),
                                                       ref="geographic_boundaries")

 
```

Does the effect of "single" method on indicator2 depend on number of maintained pops?

```{r}
msupA3<-glmmTMB(indicator2 ~ n_extant_populations + defined_populations_simplified +        n_extant_populations*defined_populations_simplified + (1|country_assessment), family = "ordbeta", data = ind2_single_methods)

# summary
summary(msupA3)
```

Because we found a relationship between method and number of populations on indicator PM, and a relationship between species range and number of populations, we further tested whether the effect of method on indicator PM is moderated by species range. 


First filter data to consider only wide ranging and restricted categories (ie remove unknown due to small sampling size)

```{r stats indicator2 and range}

## Remove unknown
data<- indicators_averaged_one  %>%
                    filter(!is.na(indicator2_mean)) %>% 
                    filter(species_range !="unknown")

# summary of indicator
summary(data$indicator2_mean)

# re-level to use geographic boundaries as reference category for the analysis
data$defined_populations_simplified<-relevel(as.factor(data$defined_populations_simplified),
                                                       ref="geographic_boundaries")

# make sure species range is a factor
data$species_range<-as.factor(data$species_range)  

```

Run model: Does method still impact indicator2 if we control for species range?

```{r}

## + country
m.b3 <- glmmTMB(indicator2_mean ~ defined_populations_simplified + species_range + (1|country_assessment), family = "ordbeta", data = data)


# summary results
summary(m.b3)

```

Similarly to the effect of number of populations on indicator 2, we further tested whether there was an interaction between method and species range, i.e. to determine whether species range was only associated with indicator 2 for some methods.

```{r}
## run model 
m.b4 <- glmmTMB(indicator2_mean ~ defined_populations_simplified + species_range + defined_populations_simplified*species_range + (1|country_assessment), family = "ordbeta", data = data)


# summary results
summary(m.b4)
```



### (c) Proportion of populations with Ne>500 (indicator1) 

Our analysis of Ne indicator followed a parallel structure to our analysis of PM indicator. 

Plot Ne indicator by method to define pops.

```{r}
# Prepare data for plot with nice labels:
# sample size of TOTAL populations
sample_size <- indicators_full %>%
                    filter(!is.na(indicator1)) %>% 
                    filter(n_extant_populations<500) %>% 
                    group_by(defined_populations_nicenames) %>% summarize(num=n())

# custom axis
## new dataframe
df<-indicators_full %>% 
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator1)) %>% 
    # add sampling size 
  left_join(sample_size) %>%
  mutate(myaxis = paste0(defined_populations_nicenames, " (n= ", num, ")")) %>%
#myaxis needs levels in the same order than defined_populations_nicenames
  mutate(myaxis = factor(myaxis, 
                  levels=levels(as.factor(myaxis))[c(1,12,2:11,13)])) # reorders levels



## plot 
pc<- df %>%
  ggplot(aes(x=myaxis, y=indicator1, color=defined_populations_nicenames,    
                                     fill=defined_populations_nicenames)) +
          geom_boxplot() + xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
          geom_jitter(size=.4, width = 0.1, color="black") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0, 0, 0, 0), "cm")) + # this is used to decrease the space between plots) 
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_x_discrete(limits=rev) +
  theme(text = element_text(size = 13))
pc

```

Scatter plot of indicator1 vs extant pops
```{r}
psupB<- indicators_full %>%
  # filter outliers with too many pops and missing data
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator1)) %>%
  filter(!is.na(n_extant_populations)) %>%
  filter(species_range !="unknown") %>%
  
  
  # plot
    ggplot(aes(x=n_extant_populations, y=indicator1, color=defined_populations_nicenames)) +
    geom_point() +
    theme_light() +
    scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
    theme(legend.position = "none") +
    ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
    xlab("Number of maintained populations") +
    theme(text = element_text(size = 13))
psupB


## Coloring by range 
psupB.1<- indicators_full %>%
  # filter outliers with too many pops and missing data
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator1)) %>%
  filter(!is.na(n_extant_populations)) %>%
  filter(species_range !="unknown") %>%
  
  
  # plot
    ggplot(aes(x=n_extant_populations, y=indicator1, color=species_range)) +
    geom_point() +
    theme_light() +
    theme(legend.position = "none") +
    ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
    xlab("Number of maintained populations") +
    theme(text = element_text(size = 13))
psupB.1

```

First we tested whether method used was associated with variation in indicator (figure c)

Prepare data for model (remove outliers and NA in desired variable) and check n:
```{r}
# remove missing data 
data_for_model<-indicators_full %>% 
                      filter(!is.na(indicator1)) %>%
                      filter(n_extant_populations<500) # doesn't make a difference in the test below, but useful for plots

# check n per method
table(data_for_model$defined_populations_simplified)

# total n
nrow(data_for_model)

# re-level to use geographic boundaries as reference category for the analysis
data_for_model$defined_populations_simplified<-relevel(as.factor(data_for_model$defined_populations_simplified),
                                                       ref="geographic_boundaries")

```

Run model asking: Does Ne indicator vary with method used? Controlling for variation in indicator among countries:  

```{r}
m.c1<-glmmTMB(indicator1 ~ defined_populations_simplified + (1|country_assessment), family = "ordbeta", data = data_for_model)
```

See results:
```{r}
summary(m.c1)
```
we next investigated whether the relationships between methods and the indicator were moderated by the role of number of populations and species range. Ie:

Does method still influence indicator1 if we control for number of populations?

```{r}
m.c2 <- glmmTMB(indicator1 ~ defined_populations_simplified + n_extant_populations + (1|country_assessment), family = "ordbeta", data = data_for_model)

summary(m.c2)
```

We then tested if there a relationship between number of maintained populations and indicator1, overall, and/or with some methods? (model associated to plot psupB)

Prepare data for model (remove outliers and NA in desired variable) and check n:
```{r}
# remove missing data 
data_for_model<-indicators_full %>% 
                      filter(!is.na(indicator1)) %>%
                      filter(!is.na(n_extant_populations)) %>%
                      filter(n_extant_populations<500) # doesn't make a difference in the test below, but useful for plots

# check number of methods
length(unique(data_for_model$defined_populations_simplified))

# check n per method
table(data_for_model$defined_populations_simplified)

# total n
nrow(data_for_model)

# re-level to use geographic boundaries as reference category for the analysis
data_for_model$defined_populations_simplified<-relevel(as.factor(data_for_model$defined_populations_simplified),
                                                       ref="geographic_boundaries")

```

Does the number of maintained pops alone affect the Ne indicator? (i.e. not controlling for method)
```{r}
msupB1<-glmmTMB(indicator1 ~ n_extant_populations + (1|country_assessment), family = "ordbeta", data= data_for_model)
```

Summary:
```{r}
summary(msupB1)
```


Does the effect of method depend on the number of populations? Or put another way, does the importance of number of populations also depend on method? 

```{r}
# run model
msupB2 <- glmmTMB(indicator1 ~ defined_populations_simplified + n_extant_populations + defined_populations_simplified*n_extant_populations + (1|country_assessment), family = "ordbeta", data = data_for_model)
```

Summary:

```{r}
summary(msupB2)
```


Because “what’s a population and how do you define them?” is such an important question, we can also test the effect of methods alone. First, subset the data to only those taxa where a single method was used: 

```{r}
ind1_single_methods<-indicators_full %>% 
                      filter(!is.na(indicator1)) %>% 
                      filter(n_extant_populations<500) %>%  # doesn't make a difference in the test below, but useful for 
                      filter(defined_populations_simplified=="genetic_clusters" | 
                             defined_populations_simplified=="geographic_boundaries" |
                             defined_populations_simplified=="eco_biogeo_proxies" | 
                             defined_populations_simplified=="management_units" |
                             defined_populations_simplified=="dispersal_buffer")


# check number of methods
length(unique(ind1_single_methods$defined_populations_simplified))

# check n by method
table(ind1_single_methods$defined_populations_simplified)

# check n total
nrow(ind1_single_methods)

# re-level to use geographic boundaries as reference category for the analysis
ind1_single_methods$defined_populations_simplified<-relevel(as.factor(ind1_single_methods$defined_populations_simplified),
                                                       ref="geographic_boundaries")

 
```

Run model:

```{r, }
# run model
msupB3 <- glmmTMB(indicator1 ~ n_extant_populations + defined_populations_simplified +        n_extant_populations*defined_populations_simplified + (1|country_assessment), family = "ordbeta", data = ind1_single_methods)
```

Summary:

```{r}
summary(msupB3)
```

Finally, we tested for associations between range type on Ne>500 indicator. 

First filter data to consider only wide ranging and restricted categories (ie remove unknown due to small sampling size)


```{r stats indicator1 and range}

## Remove unknown
data<- indicators_averaged_one  %>%
                    filter(!is.na(indicator1_mean)) %>% 
                    filter(species_range !="unknown")

# summary of indicator
summary(data$indicator1_mean)


# re-level to use geographic boundaries as reference category for the analysis
data$defined_populations_simplified<-relevel(as.factor(data$defined_populations_simplified),
                                                       ref="geographic_boundaries")

# make sure specis range is a factor
data$species_range<-as.factor(data$species_range)

```
  
  
Is there still an effect of method on indicator1 if we control for species range?


```{r}
data_full<- indicators_full  %>% # notice this uses indicators_full instead of indicators_average_one
                    filter(!is.na(indicator1)) %>% # notice this uses indicator1 instead of _mean
                    filter(species_range !="unknown")

# summary of indicator
summary(data_full$indicator1)


# re-level to use geographic boundaries as reference category for the analysis
data$defined_populations_simplified<-relevel(as.factor(data$defined_populations_simplified),
                                                       ref="geographic_boundaries")

# make sure specis range is a factor
data$species_range<-as.factor(data$species_range)

# run model
m.c3_full <- glmmTMB(indicator1 ~ defined_populations_simplified + species_range + (1|country_assessment), family = "ordbeta", data = data)


# summary results
summary(m.c3_full)
```



Finally, we tested interactions between method and species range, to determine whether the effect of species range only applies when some methods are used.


Is the effect of method on Ne indicator moderated by species range?

```{r}
data_full<- indicators_full  %>% # notice this uses indicators_full instead of indicators_average_one
                    filter(!is.na(indicator1)) %>% # notice this uses indicator1 instead of _mean
                    filter(species_range !="unknown")

## run model 
m.c4 <- glmmTMB(indicator1 ~ defined_populations_simplified + species_range + defined_populations_simplified*species_range + (1|country_assessment), family = "ordbeta", data = data_full)


# summary results
summary(m.c4)
```

Note the previous model has high Str.Errors, so this model does not converge.

### Main Figure 3 (representing the models above): Single plot 5 panels. Boxplots plots for the effect of method on: number of populations, proportion of maintained populations (indicator 2) and Proportion of populations with Ne>500 (indicator 1), AND Violin plots for the distribution of the indicator values by range type.

Top a,b,c panel boxplots:

```{r, out.height="400px", out.width="1600px"}
##### plot for Proportion of maintained populations (indicator 2) only with n in axis labels

# sample size 
sample_size <- indicators_full %>%
                    filter(!is.na(indicator2)) %>% 
                    filter(n_extant_populations<500) %>% 
                    group_by(defined_populations_nicenames) %>% summarize(num=n())

# custom axis
## new dataframe
df<-indicators_full %>% 
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator2)) %>% 
    # add sampling size 
  left_join(sample_size) %>%
  mutate(myaxis = as.factor(paste0(defined_populations_nicenames, " (n= ", num, ")")))


pb.1<- df %>%
  filter(n_extant_populations<500) %>%
  ggplot(aes(x=myaxis, y=indicator2, color=defined_populations_nicenames,    
                                     fill=defined_populations_nicenames)) +
          geom_boxplot() + xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
          geom_jitter(size=.4, width = 0.1, color="black") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0.2, 0.2, 0.2, 0.2), "cm")) + # this is used to decrease the space between plots) 
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_color_manual(values=simplifiedmethods_colors,
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  
  scale_x_discrete(limits=rev, 
                   labels= rev(sub(".*(\\(n= \\d+\\))", "\\1", levels(df$myaxis)))) + # extract "(n = number)") and show them in reverse order
  theme(text = element_text(size = 13))


##### plot for Proportion populations Ne>500 (indicator 1) only with n in axis labels
# Prepare data for plot with nice labels:
# sample size of TOTAL populations
sample_size <- indicators_full %>%
  filter(!is.na(indicator1)) %>% 
  filter(n_extant_populations<500) %>% 
  group_by(defined_populations_nicenames) %>% summarize(num=n())

# custom axis
## new dataframe
df<-indicators_full %>% 
  filter(n_extant_populations<500) %>%
  filter(!is.na(indicator1)) %>% 
  # add sampling size 
  left_join(sample_size) %>%
  mutate(myaxis = as.factor(paste0(defined_populations_nicenames, " (n= ", num, ")")))


## plot 
pc.1<- df %>%
  ggplot(aes(x=myaxis, y=indicator1, color=defined_populations_nicenames,    
             fill=defined_populations_nicenames)) +
  geom_boxplot() + xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
  geom_jitter(size=.4, width = 0.1, color="black") +
  coord_flip() +
  theme_light() +
  theme(panel.border = element_blank(), legend.position="none",
        plot.margin = unit(c(0.2, 0.2, 0.2, 0.2), "cm")) + # this is used to decrease the space between plots) 
  scale_fill_manual(values=alpha(simplifiedmethods_colors, 0.3),
                    breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_color_manual(values=simplifiedmethods_colors,
                     breaks=levels(as.factor(indicators_full$defined_populations_nicenames))) +
  scale_x_discrete(limits=rev, 
                   labels= rev(sub(".*(\\(n= \\d+\\))", "\\1", levels(df$myaxis)))) + # extract "(n = number)") and show them in reverse order
  theme(text = element_text(size = 13))


## Plot 3 panels
p_top<-plot_grid(pa, pb.1, pc.1, ncol=3, rel_widths = c(1.8,1,1), align = "h", labels=c("A)", "B)", "C)"))  
p_top

```

Bottom  d, e violin plots. Indicators by of range type coloring points to show genetic clusters

For PM indicator: 
```{r}
# add variable stating if genetic methods are used
indicators_averaged_one<- indicators_averaged_one  %>%
mutate(genetic_to_define_pops = ifelse(grepl("genetic", defined_populations_simplified), 'genetic method', 'non genetic'))


# get sample size by desired category
sample_size <- indicators_averaged_one  %>%
                    filter(!is.na(indicator2_mean)) %>% 
                    filter(!is.na(species_range)) %>% 
                    group_by(species_range) %>% summarize(num=n())

# plot
pd<-indicators_averaged_one %>% 
    filter(!is.na(indicator2_mean)) %>% 
    filter(!is.na(species_range)) %>% 
  
  # add sampling size 
  left_join(sample_size) %>% 
  mutate(myaxis = paste0(species_range, " (n= ", num, ")")) %>%

  # plot
  ggplot(aes(x=myaxis, y=indicator2_mean)) +
          geom_violin(width=1, linewidth = 0, fill="grey70")  +
          xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
          coord_flip() +
        new_scale_color() + # to color points without confuisng ggplot
        geom_jitter(size=1.2, width = 0.1, aes(color = genetic_to_define_pops)) +
        scale_color_manual(values=c("red", "black")) +
        labs(color=NULL) + # hide legend title
          theme_light() +
          theme(panel.border = element_blank(), legend.position="none", text= element_text(size=20))

```

For Ne indicator: 
```{r}

# add variable stating if genetic methods are used
indicators_averaged_one<- indicators_averaged_one  %>%
mutate(genetic_to_define_pops = ifelse(grepl("genetic", defined_populations_simplified), 'genetic method', 'non genetic'))


# get sample size by desired category
sample_size <- indicators_averaged_one  %>%
                    filter(!is.na(indicator1_mean)) %>% 
                    filter(!is.na(species_range)) %>% 
                    group_by(species_range) %>% summarize(num=n())

# plot
pe <- indicators_averaged_one %>% 
    filter(!is.na(indicator1_mean)) %>% 
    filter(!is.na(species_range)) %>% 
  
  # add sampling size 
  left_join(sample_size) %>% 
  mutate(myaxis = paste0(species_range, " (n= ", num, ")")) %>%

  # plot
  ggplot(aes(x=myaxis, y=indicator1_mean)) +
      geom_violin(width=1, linewidth = 0, fill="grey70")  +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
    new_scale_color() + # to color the points without confusing ggplot
    geom_jitter(size=1.2, width = 0.1, aes(color = genetic_to_define_pops)) +
    scale_color_manual(values=c("red", "black")) + 
    theme_light() +
    labs(color="Method to \ndefine populations") + # nicer legend title
    theme(panel.border = element_blank(), legend.position="right", text= element_text(size=20))
```

Two panel figure:

```{r}
p_bot<-plot_grid(pd + theme(legend.position = "non2"), # legend can be shown only below both plots 
          pe,
          ncol = 2,
          rel_widths = c(1,1.3), align = "h", labels=c("D)", "E)"))
```

Both top and bottom together:

```{r}
plot_grid(p_top, p_bot, ncol = 1)
ggsave("Fig3.pdf", width = 52, height = 30  , units = "cm")
```




### Supplementary Figure S7: Single figure 2 panels scatter plots number of populations vs indicators

```{r, out.height= "800px", out.width="500px"}
# plot
plot_grid(psupA + xlim(0,400) + xlab(""), # remove xlab from top plot and match x axis size
          psupB+ xlim(0,400), 
          ncol=1, align = "v", labels=c("A)", "B)"))  

ggsave("FigS7.pdf", width = 18, height = 28  , units = "cm")

```




## Indicatros by threat status (IUCN Red List)

All the following plots and analyses consider the average of multiassessed species (variable `_mean`), so that they are shown only once.

### (a) Ne > 500 indicator and red list status

Plot indicator 1 by global IUCN in the entire dataset:

```{r indicator1 gobalIUCN}

## Global IUCN

# Capitalize abbreviations 
indicators_averaged_one$global_IUCN<-as.factor(indicators_averaged_one$global_IUCN)
levels(indicators_averaged_one$global_IUCN)
indicators_averaged_one$global_IUCN<-factor(indicators_averaged_one$global_IUCN, 
                                            levels=c("cr", "en", "vu", "nt", "lc", "dd", "not_assessed","unknown"))
levels(indicators_averaged_one$global_IUCN) <- c("CR", "EN", "VU", "NT", "LC", "DD", "NE", "unknown")

## prepare data
# add sampling size
sample_size <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               filter(!is.na(global_IUCN)) %>% 
               group_by(global_IUCN) %>% summarize(num=n())

# new df 
df<- indicators_averaged_one %>% 
     filter(!is.na(indicator1_mean)) %>% 
     filter(!is.na(global_IUCN)) %>% 
        # add sampling size 
        left_join(sample_size) %>%
        mutate(myaxis = paste0(global_IUCN, " (n= ", num, ")"))


# change order of levels so that they are in the desired order
df$myaxis<-factor(df$myaxis, 
                  #grep is used below to get the sample size, which may change depending on the data
                  levels=c(grep("CR", unique(df$myaxis), value = TRUE),
                          grep("EN", unique(df$myaxis), value = TRUE),
                          grep("VU", unique(df$myaxis), value = TRUE),
                          grep("NT", unique(df$myaxis), value = TRUE),
                          grep("LC", unique(df$myaxis), value = TRUE),
                          grep("DD", unique(df$myaxis), value = TRUE),
                          grep("NE", unique(df$myaxis), value = TRUE),
                          grep("unknown", unique(df$myaxis), value = TRUE)))
      
# plot
p1<-df %>%
    ggplot(aes(x=myaxis, y=indicator1_mean , fill=global_IUCN)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
      scale_fill_manual(values= IUCNcolors, # iucn color codes
                        breaks=c(levels(df$global_IUCN))) +
      scale_x_discrete(limits=rev) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", 
            plot.title = element_text(hjust = 0.5), # center title
            text= element_text(size=15))
p1

```

Summary table:

```{r}
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               filter(!is.na(global_IUCN)) %>% 
               group_by(global_IUCN) %>% 
               summarize(n=n(),
                         mean=mean(indicator1_mean),
                         median=median(indicator1_mean),
                         per.0=sum(indicator1_mean==0) / n *100,
                         per.below.25=sum(indicator1_mean<0.25) / n *100,
                         per.below.90=sum(indicator1_mean<0.90) / n *100,
                         per.above.75=sum(indicator1_mean>0.75)/ n *100,
                         per1=sum(indicator1_mean==1) / n *100)


kable(x, digits=2)
```



Indicator 1 by country and global IUCN

```{r indicator1 country globalIUCN}


# plot
indicators_averaged_one %>% 
  filter(!is.na(regional_redlist)) %>%
  # plot
  ggplot(aes(x=global_IUCN, y=indicator1_mean, fill=global_IUCN)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
      scale_fill_manual(values= IUCNcolors, # iucn color codes
                    breaks=c(levels(indicators_averaged_one$global_IUCN))) +
      scale_x_discrete(limits=rev) +
      theme_light() +
      ggtitle("global IUCN Redlist") +
      theme(panel.border = element_blank(), legend.position="none", 
            plot.title = element_text(hjust = 0.5), # center title
            text= element_text(size=13)) +
      facet_wrap(~country_assessment, ncol = 3) +
      theme(panel.spacing = unit(1.5, "lines"))
```


### (b) Proportion of Maintained Populations and red list status?

Plot indicator 2 by global IUCN in the entire dataset:

```{r indicator2 gobalIUCN}

## Global IUCN
## prepare data
# add sampling size
sample_size <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               filter(!is.na(global_IUCN)) %>% 
               group_by(global_IUCN) %>% summarize(num=n())

# new df 
df<- indicators_averaged_one %>% 
     filter(!is.na(indicator2_mean)) %>% 
     filter(!is.na(global_IUCN)) %>% 
        # add sampling size 
        left_join(sample_size) %>%
        mutate(myaxis = paste0(global_IUCN, " (n= ", num, ")"))


# change order of levels so that they are in the desired order
df$myaxis<-factor(df$myaxis, 
                  #grep is used below to get the sample size, which may change depending on the data
                  levels=c(grep("CR", unique(df$myaxis), value = TRUE),
                          grep("EN", unique(df$myaxis), value = TRUE),
                          grep("VU", unique(df$myaxis), value = TRUE),
                          grep("NT", unique(df$myaxis), value = TRUE),
                          grep("LC", unique(df$myaxis), value = TRUE),
                          grep("DD", unique(df$myaxis), value = TRUE),
                          grep("NE", unique(df$myaxis), value = TRUE),
                          grep("unknown", unique(df$myaxis), value = TRUE)))


      
# plot
p2<-df %>%
    ggplot(aes(x=myaxis, y=indicator2 , fill=global_IUCN)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
      coord_flip() +
      scale_fill_manual(values= IUCNcolors, # iucn color codes
                        breaks=c(levels(df$global_IUCN))) +
      scale_x_discrete(limits=rev) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none", 
            plot.title = element_text(hjust = 0.5), # center title
            text= element_text(size=15))
p2

```

Summary table:
  
```{r}
x  <- indicators_averaged_one %>%
  filter(!is.na(indicator2_mean)) %>% 
  filter(!is.na(global_IUCN)) %>% 
  group_by(global_IUCN) %>% 
  summarize(n=n(),
            mean=mean(indicator2_mean),
            median=median(indicator2_mean),
            per.0=sum(indicator2_mean==0) / n *100,
            per.below.25=sum(indicator2_mean<0.25) / n *100,
            per.below.90=sum(indicator2_mean<0.90) / n *100,
            per.above.75=sum(indicator2_mean>0.75)/ n *100,
            per1=sum(indicator2_mean==1) / n *100)


kable(x, digits=2)
```

Indicator 2 by country and global IUCN

```{r indicator2 country globalIUCN}

# plot
indicators_averaged_one %>% 
  filter(!is.na(regional_redlist)) %>%
  # plot
  ggplot(aes(x=global_IUCN, y=indicator2_mean, fill=global_IUCN)) +
      geom_violin(width=1, linewidth = 0)  +
      geom_jitter(size=.5, width = 0.1) +
      xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
      coord_flip() +
      scale_fill_manual(values= IUCNcolors, # iucn color codes
                    breaks=c(levels(indicators_averaged_one$global_IUCN))) +
      scale_x_discrete(limits=rev) +
      theme_light() +
      ggtitle("global IUCN Redlist") +
      theme(panel.border = element_blank(), legend.position="none", 
            plot.title = element_text(hjust = 0.5), # center title
            text= element_text(size=13)) +
      facet_wrap(~country_assessment, ncol = 3) +
      theme(panel.spacing = unit(1.5, "lines"))
```

### Main Figure 5: Single plot 2 pannels IUCN redlist and indicator range values

```{r}
plot_grid(p2,
          p1,
          ncol=1, align = "v", labels=c("A)", "B)"))  

ggsave("Fig5.pdf", width = 20, height = 25  , units = "cm")
```


## Indicator values by taxonomic group

All the following plots and analyses consider the average of multiassessed species (variable _mean), so that they are shown only once. 

We also grouped taxa with small n (<5) into "others", according to the following table:

```{r}
table(indicators_averaged_one$taxonomic_group)
```

They are grouped along with "other" in a new category "others" in the new variable `taxonomic_group_simplified`:

```{r}
indicators_averaged_one <- indicators_averaged_one %>% 
                           ungroup() %>% 
  mutate(taxonomic_group_simplified = case_when(
                                       # if the taxon group is in the list of groups with small n change to "others"
                                       as.character(taxonomic_group) %!in% c("bryophyte", "fungus", "other") ~  as.character(taxonomic_group),
                                       TRUE ~ "others"))

# check:
table(indicators_averaged_one$taxonomic_group_simplified)
```

We also create a group of only 3 categories for animals, plants and others:

```{r}
# Define the grouping map
grouping_map <- c(
  "amphibian", "bird", "fish", "invertebrate", "mammal",
  "angiosperm", "gymnosperm", "reptile", "pteridophytes", "others"
)

# Create a new variable taxonomic_group_3
indicators_averaged_one <- indicators_averaged_one %>%
                            mutate(
                              taxonomic_group_3 = case_when(
                                taxonomic_group_simplified %in% grouping_map[1:5] ~ "animals",
                                taxonomic_group_simplified %in% grouping_map[6:9] ~ "plants",
                                taxonomic_group_simplified %in% grouping_map[10] ~ "others",
                                TRUE ~ NA_character_
                              )
                            )

# reorder levels
indicators_averaged_one$taxonomic_group_3<- factor(indicators_averaged_one$taxonomic_group_3, 
                                                  levels=c("animals", "plants", "others"))
```


### Violin plots, histograms and summary tables for each indicator by taxonomic group

**Indicator Ne > 500**

```{r}
## prepare data
# add sampling size
sample_size <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               group_by(taxonomic_group_simplified) %>% summarize(num=n())

# new df 
df<- indicators_averaged_one %>% 
     filter(!is.na(indicator1_mean)) %>% 
        # add sampling size 
        left_join(sample_size) %>%
        mutate(myaxis = paste0(taxonomic_group_simplified, " (n= ", num, ")"))

# change order of levels so that they are in the desired order
df$myaxis<-factor(df$myaxis, 
                  #grep is used below to get the sample size, which may change depending on the data
                    levels=c(grep("amphibian", unique(df$myaxis), value = TRUE), 
                             grep("bird" , unique(df$myaxis), value = TRUE),
                             grep("fish" , unique(df$myaxis), value = TRUE),
                             grep("invertebrate", unique(df$myaxis), value = TRUE),
                             grep("mammal", unique(df$myaxis), value = TRUE),
                             grep("reptile", unique(df$myaxis), value = TRUE),
                             grep("angiosperm", unique(df$myaxis), value = TRUE),
                             grep("gymnosperm", unique(df$myaxis), value = TRUE),
                             grep("pteridophytes", unique(df$myaxis), value = TRUE),
                             grep("others" , unique(df$myaxis), value = TRUE)))

df$taxonomic_group_simplified<-factor(df$taxonomic_group_simplified, 
                                       levels=c("amphibian", "bird" , "fish" , "invertebrate", "mammal", "reptile",
                                                 "angiosperm",  "gymnosperm", "pteridophytes",
                                                 "others"))

        
# plot
p1<-df %>%
    ggplot(aes(x=myaxis, y=indicator1_mean, fill=taxonomic_group_simplified, color=taxonomic_group_simplified)) +
      geom_violin(width=1.5, linewidth = 0.2)  +
      geom_jitter(size=.7, width = 0.1, color="black") +
      xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
      coord_flip() +
      scale_x_discrete(limits=rev) +
      scale_fill_manual(values= c(rep(grouped_taxon_colors[1], 6), # for animals
                                  rep(grouped_taxon_colors[2], 3), # for platns
                                  rep(grouped_taxon_colors[3], 1)), # for fungi and others
                       breaks=c(levels(df$taxonomic_group_simplified))) +
      scale_color_manual(values= c(rep(grouped_taxon_colors[1], 6), # for animals
                                  rep(grouped_taxon_colors[2], 3), # for platns
                                  rep(grouped_taxon_colors[3], 1)), # for fungi and others
                       breaks=c(levels(df$taxonomic_group_simplified))) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none",
            text= element_text(size=15))
p1
```


Table with sampling size, mean indicator value and proporiton of taxa where the value is below 0.25, 0.50 and 0.75:

```{r}
#summary table by taxonomic group
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               filter(!is.na(taxonomic_group_simplified)) %>% 
               group_by(taxonomic_group_simplified) %>% 
               summarize(n=n(),
                         mean=mean(indicator1_mean),
                         median=median(indicator1_mean),
                         n.below.75=sum(indicator1_mean<0.75),
                         n.below.50=sum(indicator1_mean<0.50),
                         n.below.25=sum(indicator1_mean<0.25),
                         per.below.25=n.below.25/n*100,
                         per.below.50=n.below.50/n*100)

# Calculate total counts and means
total_counts <- summarise(x,
                          taxonomic_group_simplified = "ALL",
                          n = sum(n),
                          mean= mean(mean),
                          median=median(median),
                          n.below.75 = sum(n.below.75),
                          n.below.50 = sum(n.below.50),
                          n.below.25 = sum(n.below.25),
                          per.below.25 = n.below.25 / n * 100,
                          per.below.50 = n.below.50 / n * 100)

# Bind the total row to the summary_table
summary_table <- bind_rows(x, total_counts)

# keep taxonomic groups as level in desired order:
summary_table$taxonomic_group_simplified<-factor(summary_table$taxonomic_group_simplified,
                                                 levels = c("amphibian", "bird" , "fish" , "invertebrate", "mammal",
                                                 "angiosperm", "gymnosperm",  "reptile", "pteridophytes",
                                                "others", "ALL"))
summary_table<- summary_table %>% arrange(taxonomic_group_simplified)

# show nice table
kable(summary_table, digits=2)
```



**Indicator Proportion of maintained populations:**

```{r}
## prepare data
# add sampling size
sample_size <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               group_by(taxonomic_group_simplified) %>% summarize(num=n())

# new df 
df<- indicators_averaged_one %>% 
     filter(!is.na(indicator2_mean)) %>% 
        # add sampling size 
        left_join(sample_size) %>%
        mutate(myaxis = paste0(taxonomic_group_simplified, " (n= ", num, ")"))

# change order of levels so that they are in the desired order
df$myaxis<-factor(df$myaxis, 
                  #grep is used below to get the sample size, which may change depending on the data
                    levels=c(grep("amphibian", unique(df$myaxis), value = TRUE), 
                             grep("bird" , unique(df$myaxis), value = TRUE),
                             grep("fish" , unique(df$myaxis), value = TRUE),
                             grep("invertebrate", unique(df$myaxis), value = TRUE),
                             grep("mammal", unique(df$myaxis), value = TRUE),
                             grep("reptile", unique(df$myaxis), value = TRUE),
                             grep("angiosperm", unique(df$myaxis), value = TRUE),
                             grep("gymnosperm", unique(df$myaxis), value = TRUE),
                             grep("pteridophytes", unique(df$myaxis), value = TRUE),
                             grep("others" , unique(df$myaxis), value = TRUE)))

df$taxonomic_group_simplified<-factor(df$taxonomic_group_simplified, 
                           levels=c("amphibian", "bird" , "fish" , "invertebrate", "mammal",  "reptile",
                                     "angiosperm",  "gymnosperm", "pteridophytes",
                                     "others"))

        
# plot
p2<-df %>%
    ggplot(aes(x=myaxis, y=indicator2_mean, fill=taxonomic_group_simplified, color=taxonomic_group_simplified)) +
      geom_violin(width=1, linewidth = 0.2)  +
      geom_jitter(size=.7, width = 0.1, color="black") +
      xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
      coord_flip() +
      scale_x_discrete(limits=rev) +
      scale_fill_manual(values= c(rep(grouped_taxon_colors[1], 6), # for animals
                                  rep(grouped_taxon_colors[2], 3), # for platns
                                  rep(grouped_taxon_colors[3], 1)), # for fungi and others
                       breaks=c(levels(df$taxonomic_group_simplified))) +
      scale_color_manual(values= c(rep(grouped_taxon_colors[1], 6), # for animals
                                  rep(grouped_taxon_colors[2], 3), # for platns
                                  rep(grouped_taxon_colors[3], 1)), # for fungi and others
                       breaks=c(levels(df$taxonomic_group_simplified))) +
      theme_light() +
      theme(panel.border = element_blank(), legend.position="none",
            text= element_text(size=15))
p2
```

Table with sampling size, mean indicator value and proporiton of taxa where the value is below 0.25, 0.50 and 0.75:


```{r}
# summary table for taxonomic group:
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               filter(!is.na(taxonomic_group_simplified)) %>% 
               group_by(taxonomic_group_simplified) %>% 
               summarize(n=n(),
                         mean=mean(indicator2_mean),
                         median=median(indicator2_mean),
                         n.below.75=sum(indicator2_mean<0.75),
                         n.below.50=sum(indicator2_mean<0.50),
                         n.below.25=sum(indicator2_mean<0.25),
                         per.below.25=n.below.25/n*100,
                         per.below.50=n.below.50/n*100)


# Calculate total counts and means
total_counts <- summarise(x,
                          taxonomic_group_simplified = "ALL",
                          n = sum(n),
                          mean = mean(mean),
                          median = median(median),
                          n.below.75 = sum(n.below.75),
                          n.below.50 = sum(n.below.50),
                          n.below.25 = sum(n.below.25),
                          per.below.25 = n.below.25 / n * 100,
                          per.below.50 = n.below.50 / n * 100)

# Bind the total row to the summary_table
summary_table <- bind_rows(x, total_counts)

# keep taxonomic groups as level in desired order:
summary_table$taxonomic_group_simplified<-factor(summary_table$taxonomic_group_simplified,
                                                 levels = c("amphibian", "bird" , "fish" , "invertebrate", "mammal",
                                                 "angiosperm", "gymnosperm",  "reptile", "pteridophytes",
                                                "others", "ALL"))
summary_table<- summary_table %>% arrange(taxonomic_group_simplified)

# show nice table
kable(summary_table, digits=2)

```

### Histograms and summary tables by 3 taxonomic groups (animals, plants, others)

By animals, plants, others:

```{r}
# Create a histogram 
hist_p1 <- indicators_averaged_one %>%
                  ggplot(aes(x = indicator1_mean, fill = taxonomic_group_3)) +
                  geom_histogram( bins = 25, color="white") + # Adjust the number of bins as needed
                  labs(x = "Proportion of populations within species with Ne>500 \n(Ne 500 indicator)", y = "Frequency") +
                  scale_fill_manual(
                    values = grouped_taxon_colors, # Custom colors for animals, plants, and others
                    breaks = c("animals", "plants", "others"),
                    name = "Taxonomic Group")+
                  theme_light() +
                  theme(panel.border = element_blank(), text = element_text(size = 15), 
                        legend.position = "right") +
                  guides(fill = guide_legend(title = NULL))

# plot
hist_p1
```
Summary table for Ne indicator 3 taxonomic groups:

```{r}
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               filter(!is.na(taxonomic_group_3)) %>% 
               group_by(taxonomic_group_3) %>% 
               summarize(n=n(),
                         mean=mean(indicator1_mean),
                         median=median(indicator1_mean),
                         per.0=sum(indicator1_mean==0) / n *100,
                         per.below.25=sum(indicator1_mean<0.25) / n *100,
                         per.below.90=sum(indicator1_mean<0.90) / n *100,
                         per.above.75=sum(indicator1_mean>0.75)/ n *100,
                         per1=sum(indicator1_mean==1) / n *100)



# Calculate total counts and means
total_counts <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               filter(!is.na(taxonomic_group_3)) %>% 
               ungroup() %>% 
               summarize(taxonomic_group_3 = "ALL",
                          n= n(),
                          mean = mean(indicator1_mean),
                          median = median(indicator1_mean),
                          per.0=sum(indicator1_mean==0) / n *100,
                          per.below.25=sum(indicator1_mean<0.25) / n *100,
                          per.below.90=sum(indicator1_mean<0.90) / n *100,
                          per.above.75=sum(indicator1_mean>0.75)/ n *100,
                          per1=sum(indicator1_mean==1) / n *100)

# Bind the total row to the summary_table
summary_table <- bind_rows(x, total_counts)

# keep taxonomic groups as level in desired order:
summary_table$taxonomic_group_3<-factor(summary_table$taxonomic_group_3,
                                                 levels = c("animals", "plants", "others", "ALL"))
summary_table<- summary_table %>% arrange(taxonomic_group_3)

kable(summary_table, digits=2)
```


PM  Histogram for animal, plants, others:

```{r}
# Create a histogram 
hist_p2 <- indicators_averaged_one %>%
                  ggplot(aes(x = indicator2_mean, fill = taxonomic_group_3)) +
                  geom_histogram(bins = 25, color="white") + # Adjust the number of bins as needed
                  labs(x = "Proportion of populations maintained within species \n(PM indicator)", y = "Frequency") +
                  scale_fill_manual(
                    values = grouped_taxon_colors, # Custom colors for animals, plants, and others
                    breaks = c("animals", "plants", "others"),
                    name = "Taxonomic Group")+
                  theme_light() +
                  theme(panel.border = element_blank(), text = element_text(size = 15)) +
                  guides(fill = guide_legend(title = NULL))

# plot
hist_p2
```

Summary table for PM indicator 3 taxonomic groups

```{r}
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               filter(!is.na(taxonomic_group_3)) %>% 
               group_by(taxonomic_group_3) %>% 
               summarize(n=n(),
                         mean=mean(indicator2_mean),
                         median=median(indicator2_mean),
                         per0=sum(indicator2_mean==0) / n *100,
                         per.below.25=sum(indicator2_mean<0.25) / n *100,
                         per.below.90=sum(indicator2_mean<0.90) / n *100,
                         per.above.75=sum(indicator2_mean>0.75) / n *100,
                         per1=sum(indicator2_mean==1) / n *100)

# Calculate total counts and means
total_counts <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               filter(!is.na(taxonomic_group_3)) %>% 
               ungroup() %>% 
               summarize(taxonomic_group_3 = "ALL",
                          n= n(),
                          mean = mean(indicator2_mean),
                          median = median(indicator2_mean),
                          per0=sum(indicator2_mean==0) / n *100,
                          per.below.25=sum(indicator2_mean<0.25) / n *100,
                          per.below.90=sum(indicator2_mean<0.90) / n *100,
                          per.above.75=sum(indicator2_mean>0.75) / n *100,
                          per1=sum(indicator2_mean==1) / n *100)

# Bind the total row to the summary_table
summary_table <- bind_rows(x, total_counts)

# keep taxonomic groups as level in desired order:
summary_table$taxonomic_group_3<-factor(summary_table$taxonomic_group_3,
                                                 levels = c("animals", "plants", "others", "ALL"))
summary_table<- summary_table %>% arrange(taxonomic_group_3)

kable(summary_table, digits=2)
```


### Main Figure 4: Single figure 4 panels for violin plots and histograms for both indicators by taxonomic group

```{r}
plot_grid(p2, hist_p2, 
          p1, hist_p1,
          ncol=2, align = "v", labels=c("A)", "B)", "C)", "D)"))

ggsave("Fig4.pdf", width = 37, height = 25  , units = "cm")
```

## Supplementary Figure S2: Values of indicator 1 and indicator 2 for multiassessed taxa (alternative assessments)

```{r}
#subset only with taxa assessed multiple times:
only_multi<-indicators_full %>% 
                          filter(multiassessment=="multiassessment") 
```

First, check how indicator 1 changes across the multiassessments.

```{r}
p1<-only_multi %>% 
  # Keep rows with different values in indicator1 within each taxon group
  group_by(taxon) %>%
  filter(n_distinct(indicator1) > 1) %>%
  # plot
  ggplot(aes(x=taxon, y=indicator1)) +
          geom_line(colour="darkgrey") + 
          geom_point(aes(color=country_assessment)) +
  xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
  labs(color="country") +
  ylim(0, 1)+
  coord_flip() +
  theme_light() + 
  theme(axis.text.y = element_text(face = "italic"),  panel.border = element_blank(), legend.position="right", text= element_text(size=13))
p1

```

Now check how Proportion of maintained populations (indicator 2) changes across the multiassessments.

```{r}
p2<-only_multi %>% 
  # Keep rows with different values in indicator1 within each taxon group
  group_by(taxon) %>%
  filter(n_distinct(indicator2) > 1) %>%
  
  ggplot(aes(x=taxon, y=indicator2)) +
          geom_line(colour="darkgrey") + 
          geom_point(aes(color=country_assessment)) +
    scale_color_manual(values= scales::hue_pal()(4)[2:4]) + # last 3 colors to make them the same than the other plot
  xlab("") + ylab("Proportion of populations maintained within species \n(PM indicator)") +
  labs(color="country") +
  coord_flip() +
  theme_light() + 
  theme(axis.text.y = element_text(face = "italic"), panel.border = element_blank(), legend.position="right", text= element_text(size=13))
p2
```

Plot together:
```{r}
plot_grid(p2, p1,  
          rel_heights = c(1.4, 0.8),
          ncol=1, labels=c("A)", "B)")) 

ggsave("FigS2.pdf", width = 20, height = 25  , units = "cm")
```



## Indicator 3 (number of species with genetic diversity monitoring)
Indicator 3 refers to the number (count) of taxa by country in which genetic monitoring is occurring. This is stored in the variable temp_gen_monitoring as a “yes/no” answer for each taxon.

```{r}
indicator3
```

Plot by global IUCN redlist status
```{r}
# desired order of levels
ind3_data$global_IUCN<-factor(as.factor(indicators_full$global_IUCN), levels=c("cr", "en", "vu", "nt", "lc", "dd", "not_assessed", "unknown"))


## plot
ind3_data %>%
                 # keep only one record if the taxon was assessed more than once within the country
                 select(country_assessment, taxon, temp_gen_monitoring, global_IUCN) %>%
                 filter(!duplicated(.)) %>%

                 # count "yes" in tem_gen_monitoring by country
                 filter(temp_gen_monitoring=="yes") %>%
ggplot(aes(x=country_assessment, fill=global_IUCN)) +
  geom_bar() +
  xlab("") + ylab("Number of taxa with temporal genetic diversity monitoring") +
  scale_fill_manual(values= IUCNcolors, # iucn color codes
                    breaks=levels(as.factor(indicators_full$global_IUCN))) +
      theme_light()
```

### Supplementary Figure S9: Sankey plot genetic data availability
Relatively few taxa have genetic monitoring, but many have some sort of genetic study. Let’s check that with a Sankey Plot:

```{r}
# check original n of assessments
nrow(ind3_data)

# first subset the ind3_data keeping only taxa assessed a single time, plust the first record of those assessed multiple times.
ind3_data_firstmulti<-ind3_data[!duplicated(cbind(ind3_data$taxon, ind3_data$country_assessment)), ]

# n taxa 
nrow(ind3_data_firstmulti)

# transform data to how ggsankey wants it
df <- ind3_data_firstmulti %>%
  make_long(country_assessment, temp_gen_monitoring, gen_studies)

# plot
ggplot(df, aes(x = x,
               next_x = next_x,
               node = node,
               next_node = next_node,
               fill = factor(node),
               label = node)) +
  geom_sankey(flow.alpha = 0.5, 
              show.legend = FALSE) +
  geom_sankey_label(size = 2.5, color = "black", fill = "white") +
  theme_sankey(base_size = 10) +

    # manually set flow fill according to desired color
                            # countries
  scale_fill_manual(values=c(scales::hue_pal()(length(unique(ind3_data_firstmulti$country_assessment))),  
                             # traffic light for monitoring
                             c("darkolivegreen", "brown3", "darkgrey"),
                             # nice soft colors for gen_studies
                             c("grey50", "grey35", "grey50", "brown3")),
                              
                    breaks=c(unique(ind3_data_firstmulti$country_assessment),
                             unique(ind3_data_firstmulti$temp_gen_monitoring),
                             unique(ind3_data_firstmulti$gen_studies))) +
  
  xlab("")

ggsave("FigS9.pdf", width = 20, height = 18  , units = "cm")

```

```{r}
table(ind3_data_firstmulti$gen_studies)
```


Count data:
```{r}
ind3_data %>%
                 # keep only one record if the taxon was assessed more than once within the country
                 select(country_assessment, taxon, gen_studies, temp_gen_monitoring) %>%
                 filter(!duplicated(.)) %>%

                 group_by(country_assessment, temp_gen_monitoring, gen_studies) %>%
                 summarise(n_studies=n())
```

How many genetic studies ara available by country for species without temporal genetic diversity monitoring?
```{r}
## plot
ind3_data %>%
                 # keep only one record if the taxon was assessed more than once within the country
                 select(country_assessment, taxon, temp_gen_monitoring, gen_studies) %>%
                 filter(!duplicated(.)) %>%
                 # keep only taxa without gen div monitoring
                 filter(temp_gen_monitoring=="no")%>%

ggplot(aes(x=country_assessment, fill=gen_studies)) +
  geom_bar() +
    scale_fill_manual(values=c("grey80", scales::hue_pal()(3)))+
  xlab("")  +
      theme_light()
```

## Summary table of mean indicator values and n

The tables below show the indicator values and sampling size averaging them by country, taxonomic group, distribution type or IUCN global red list status. For this summary the mean of the multiassessed species was considering and counted as a single entry for the sampling size. 

Codes for indicator names:

- **PM.ind**: Proportion of Mantained populations indicator (indicator 2)
- **Ne.ind**: Proportion of populations where Ne>500 indicator (indicator 1)
- **Mon.ind**: Number of species where genetic diversity monitoring is taking place (indicator 3)

Codes for summary stats:

- **n**: sampling size (number of taxa assessed) without missing data
- **mean**: mean value for the indicator value
- **sd**: standar deviation for the indicator value

Summary stats by country:
```{r}
x<-indicators_averaged_one %>% 
                group_by(country_assessment) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)

```


### Taxonomic groups

Summary stats by taxonomic group:

```{r}
x<-indicators_averaged_one %>% 
                group_by(taxonomic_group) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)

```

Detailed table:

```{r}
x<-indicators_averaged_one %>% 
                group_by(country_assessment, taxonomic_group) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)
```


### IUCN 

Summary stats:

```{r}
x<-indicators_averaged_one %>% 
                group_by(global_IUCN) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)

```

Detailed table by IUCN category:

```{r}
x<-indicators_averaged_one %>% 
                group_by(country_assessment, global_IUCN) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)
```


### Distribution type

Summary stats:

```{r}
x<-indicators_averaged_one %>% 
                group_by(species_range) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)

```

Detailed table by IUCN category:

```{r}
x<-indicators_averaged_one %>% 
                group_by(country_assessment, species_range) %>%
                summarise(n.PM.ind=sum(!is.na(indicator2)), 
                          mean.PM.ind=mean(indicator2, na.rm=TRUE),
                          sd.PM.ind=sd(indicator2, na.rm=TRUE),
                          n.Ne.ind=sum(!is.na(indicator1)), 
                          mean.Ne.ind=mean(indicator1, na.rm=TRUE),
                          sd.Ne.ind=sd(indicator1, na.rm=TRUE))

# nice table
kable(x, digits=3)
```

## 

## Simplified figures and basic stats for text summary and policy brief

### How many species and pops:

How many species:
```{r}
nrow(indicators_averaged_one)
```

How many assessments (including species assessed more than once):
```{r}
nrow(indicators_full)
```

How many populations, including all pops from species that were assessed more than once:

```{r}
nrow(ind1_data)
```

How many populations by country (including multiassesments):
```{r}
npops<-ind1_data %>%
    group_by(country_assessment) %>%
    summarise(n_pops=n())
npops
mean(npops$n_pops)
sd(npops$n_pops)
```


How many populations, counting only once populations from taxa assessed more than once:

```{r}
# This looks for the id of the taxa keeping only 1 for the multiassessed taxa, and keeps those int he ind1_data (where the pops data is)
ind1_data_without_altassesments<-ind1_data[ind1_data$X_uuid %in% indicators_averaged_one$X_uuid, ]

# the number of rows is the number of pops counting only once multiassessed taxa
nrow(ind1_data_without_altassesments)

```


How many multiassesments:
```{r}
sum(indicators_full$multiassessment=="multiassessment")

```

Which taxa had multiassesments and how many:
```{r}
x<- indicators_full %>% filter(multiassessment=="multiassessment") %>%
                    group_by(taxon) %>%
                    summarise(n=n())
kable(x)
```
How many taxa with multiassesments?
```{r}
nrow(x)
```

#### Fill gaps in Table S1 
These numbers are not reported in the main text, but it is useful to have them in Table S1 to better understand the dataset.

Number of **taxa** with No data on PM indicator in both of the alternative assesments. 
```{r}
sum(is.na(indicators_averaged_one$indicator2_mean))
```

Percentage out of `r nrow(indicators_averaged_one)`:
```{r}
sum(is.na(indicators_averaged_one$indicator2_mean))/nrow(indicators_averaged_one)
```


Number of **taxa** with data on Ne 500 indicator (for at least 1 pop) in both of the alternative assesments. 

```{r}
sum(!is.na(indicators_averaged_one$indicator1_mean))
```


Percentage out of `r nrow(indicators_averaged_one)`:
```{r}
sum(!is.na(indicators_averaged_one$indicator1_mean))/nrow(indicators_averaged_one)
```

**Number of populations data, considering only one of the alternative assessments**

Number of pops:
```{r}
nrow(ind1_data_without_altassesments)
```


```{r}
df<-ind1_data_without_altassesments %>%
   mutate(Ne_calculated_from = replace_na(Ne_calculated_from, "no data available")) %>%
   group_by(Ne_calculated_from) %>%
   summarise(n=n(),
             percentage = (n / nrow(ind1_data_without_altassesments)) * 100)

kable(df, digits = 0)
```



### Plain Histogram and stats for Ne > 500 indicator 

Plain histogram:

```{r}
# Create a histogram 
hist_p <- indicators_averaged_one %>%
                  ggplot(aes(x = indicator1_mean)) +
                  geom_histogram( bins = 25, fill="grey30") + # Adjust the number of bins as needed
                  labs(x = "Proportion of populations within species with Ne>500 \n(Ne 500 indicator)", y = "Frequency") +
                  theme_light() +
                  theme(panel.border = element_blank(), text = element_text(size = 15)) +
                  guides(fill = guide_legend(title = NULL))

# plot
hist_p
```

Summary stats for the Ne 500 indicator:
```{r}
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator1_mean)) %>% 
               ungroup() %>% 
               summarize(n=n(),
                         mean=mean(indicator1_mean),
                         median=median(indicator1_mean),
                         per.0=sum(indicator1_mean==0) / n *100,
                         per.below.25=sum(indicator1_mean<0.25) / n *100,
                         per.below.90=sum(indicator1_mean<0.90) / n *100,
                         per.above.75=sum(indicator1_mean>0.75)/ n *100,
                         per1=sum(indicator1_mean==1) / n *100)
x
kable(x, digits = 2)
```

Data availability for the Ne indicator. At the species level:
```{r}
sum(!is.na(indicators_averaged_one$indicator1_mean)) / nrow(indicators_averaged_one)
```

At the population level:

```{r}
sum(!is.na(ind1_data$Ne_combined)) / nrow(ind1_data)
```


Populations below the Ne 500 threshold

```{r}
x<- ind1_data %>% 
  ungroup() %>% 
  summarise(n_pops = n(), 
            n_pops_Ne_data = sum(!is.na(Ne_combined)), 
            n_pops_more_500 = sum(Ne_combined >= 500, na.rm = TRUE),
            n_pops_less_500 =sum(Ne_combined < 500, na.rm = TRUE),
            per_less_500 = n_pops_less_500/n_pops_Ne_data)
kable(x, digits=2)
```


### Plain Histogram and stats for Proportion Mantained populations

Plain histogram

```{r}
# Create a histogram 
hist_p <- indicators_averaged_one %>%
                  ggplot(aes(x = indicator2_mean)) +
                  geom_histogram(bins = 25, fill="grey30") + # Adjust the number of bins as needed
                  labs(x = "Proportion of populations maintained within species \n(PM indicator)", y = "Frequency") +
                  theme_light() +
                  theme(panel.border = element_blank(), text = element_text(size = 15)) +
                  guides(fill = guide_legend(title = NULL))

# plot
hist_p
```

Summary stats for the PM indicator:

```{r}
x  <- indicators_averaged_one %>%
               filter(!is.na(indicator2_mean)) %>% 
               ungroup() %>% 
               summarize(n=n(),
                         mean=mean(indicator2_mean),
                         median=median(indicator2_mean),
                         per0=sum(indicator2_mean==0) / n *100,
                         per.below.25=sum(indicator2_mean<0.25) / n *100,
                         per.below.90=sum(indicator2_mean<0.90) / n *100,
                         per.above.75=sum(indicator2_mean>0.75) / n *100,
                         per1=sum(indicator2_mean==1) / n *100)

kable(x, digits = 2)
```


### Data availability, donuts and plot bars for Ne 500


Species level yes/no table with percentages for Ne 500 indicator
```{r}
df<- indicators_full %>%
     group_by(popsize_data) %>%
   summarise(n=n(),
             percentage = (n / nrow(metadata)) * 100)
   
kable(df, digits = 0)

```

Donut only available data

```{r}
df<- indicators_full %>%
     filter(popsize_data != "data_for_species") %>% # we want to show only data for pops or insufficient
     group_by(popsize_data) %>%
   summarise(n=n(),
             percentage = (n / nrow(metadata)) * 100)

# variable to make change the size of the hole
hsize <- 2 # to change the size of the hole. larger=bigger 
df <- df %>% 
  mutate(x = hsize)  

# donut plot
p <- ggplot(df, aes(x = hsize, y = n, fill = popsize_data)) +
  geom_col() +
  coord_polar(theta = "y") +
  scale_fill_manual(values=c("#2ca02c", "grey80"),
                    breaks=c("yes", "insuff_data_species"),
                    labels=c("Population level", "Insufficient data")) +

  xlim(c(0.2, hsize + 0.5)) + theme_void()
p

```




Species level yes/no. Bar plot for Ne 500
```{r}
indicators_full %>%
     filter(popsize_data != "data_for_species") %>% # we want to show only data for pops or insufficient
      ggplot(aes(x=country_assessment, fill = popsize_data)) +
      geom_bar(position = "fill", color="white") +
      scale_fill_manual(values=c("#2ca02c", "grey80"),
                        breaks=c("yes", "insuff_data_species"),
                        labels=c("Population level", "Insufficient data")) +
      scale_x_discrete(limits=rev) + xlab("") + ylab("Data availability (% of species)") +
      coord_flip() +
      theme_light()


```


Population level, what kind? Table
```{r}
# we first need the column numbers
df<-ind1_data %>%
   mutate(Ne_calculated_from = replace_na(Ne_calculated_from, "no data available")) %>%
   group_by(Ne_calculated_from) %>%
   summarise(n=n(),
             percentage = (n / nrow(ind1_data)) * 100)
   
kable(df, digits = 0)

```
Donut
```{r}
# variable to make change the size of the hole
hsize <- 3 # to change the size of the hole. larger=bigger 
df <- df %>% 
  mutate(x = hsize)  

# donut plot
p <- ggplot(df, aes(x = hsize, y = n, fill = Ne_calculated_from)) +
  geom_col() +
  coord_polar(theta = "y") +
  scale_fill_manual(labels=c("genetic data", "NcPoint ratio", "NcRange ratio", "no data available"),
                      breaks=c("genetic data", "NcPoint ratio", "NcRange ratio", "no data available"),
                      values=c("darkgreen", "#0072B2", "#E69F00", "grey80")) +
  xlim(c(0.2, hsize + 0.5)) + theme_void()
p

```





### Data availability for PM indicator

Total taxa with NA in extinct populations:
```{r}
sum(is.na(indicators_full$n_extint_populations))
```

Percentage of missing data
```{r}
sum(is.na(indicators_full$n_extint_populations))/nrow(indicators_full)
```

Total taxa with data availability on extinct pops
```{r}
sum(!is.na(indicators_full$n_extint_populations))
```

Percentage of taxa with data availability on extinct pops (which also includes NA for extant, see above)
```{r}
sum(!is.na(indicators_full$n_extint_populations))/nrow(indicators_full)

nrow(indicators_full)
```


### Data availability for at least one indicator

Data availability for at least one indicator. Including multiassesments
```{r}
# number
x<- indicators_full %>%
     filter(popsize_data=="yes" | !is.na(n_extint_populations))
nrow(x)

# percentage
nrow(x) / nrow(indicators_full)
```

Data availability for at least one indicator. Keeping only one of the multiassesments
```{r}
# number
x<- indicators_averaged_one %>%
     filter(popsize_data=="yes" | !is.na(n_extint_populations))
nrow(x)

# percentage
nrow(x) / nrow(indicators_averaged_one)
```

## Exploring Ne 500 values if using a different Ne/Nc ratio

Because the Ne/Nc does not necessarely needs to be 0.1, wenterested to know if the distribution of values shifts to the right (better indicator status) if we used other ratio, like 0.2 or 0.3, which is a better ratio for some taxa. This would show uncertainty and emphasize more work to be done on the Ne/Nc ratio. Lastly it may help address concerns that Nc 5000 is just infeasible, because there are outlier species which never had Nc>5000, e.g. some endimic birds, but these are cases where Ne might be >500 (because Ne/Nc is >0.25).

To address this, we:

1) Explore the NcPoint data converting it to Ne with a ratio of 0.2 and 0.3 and examine the indicator value
2) Explore the differences in the indicator value of the taxa for which we have a different Ne/Nc ratio.

### Ne 500 indicator using 0.2 and 03 on all taxa with NcPoint data

First we get the indicator 1 data again from scratch (because the object ind1_data already has the transformed Ne etc)

```{r}
# ind1 data form kobo
data_for_alternative_ratio<-get_indicator1_data(kobo_output=kobo_clean)
# bind data form templates
data_for_alternative_ratio<-rbind(data_for_alternative_ratio, ind1_data_from_templates)
```
Keep only populations for which we hav NcPoint data:
```{r}
# filter rows with missing data for NcPoint
data_for_alternative_ratio<-data_for_alternative_ratio %>% 
                            filter(!is.na(NcPoint))
# show data to check
data_for_alternative_ratio %>% select(taxon, Ne, NcRange, NcPoint)
```
How many pops and assessments this subset of data has?
```{r}
# How many pops?
nrow(data_for_alternative_ratio)

# how many assesments?
length(unique(data_for_alternative_ratio$X_uuid))

# how many taxa?
length(unique(data_for_alternative_ratio$taxon))

# how many assessments by taxonomic groups
data_for_alternative_ratio %>% group_by(X_uuid, taxonomic_group) %>%
                      summarise(n=n()) %>% group_by(taxonomic_group)  %>%
                      summarise(n=n())

# how many assessments by country
data_for_alternative_ratio %>% group_by(X_uuid, country_assessment) %>%
                      summarise(n=n()) %>% group_by(country_assessment)  %>%
                      summarise(n=n())
```


Now we transform NcPoint data to Ne using 0.1, 0.2 and 0.3 ratios

```{r}
data_for_alternative_ratio <- data_for_alternative_ratio %>%
                              mutate(
                               Ne_0.1=NcPoint*0.1,
                               Ne_0.2=NcPoint*0.2,
                               Ne_0.3=NcPoint*0.3)
# show data to check
data_for_alternative_ratio %>% select(NcPoint, Ne_0.1, Ne_0.2, Ne_0.3)

```
Estimate the Ne 500 indciator for each of the 3 Ne values 

```{r}
ind1_alt_ratios<-data_for_alternative_ratio %>%
                  group_by(X_uuid, taxonomic_group, taxon) %>%
                  summarise(n_pops=n(),
                            n_pops_Ne_data=sum(!is.na(NcPoint)),
                            n_more_500_Ne_0.1=sum(Ne_0.1>500, na.rm=TRUE),
                            n_more_500_Ne_0.2=sum(Ne_0.2>500, na.rm=TRUE),
                            n_more_500_Ne_0.3=sum(Ne_0.3>500, na.rm=TRUE),
                            ind1_Ne_0.1=n_more_500_Ne_0.1/n_pops_Ne_data,
                            ind1_Ne_0.2=n_more_500_Ne_0.2/n_pops_Ne_data,
                            ind1_Ne_0.3=n_more_500_Ne_0.3/n_pops_Ne_data) %>%
                            ungroup()


ind1_alt_ratios %>% select(taxonomic_group, taxon, ind1_Ne_0.1, ind1_Ne_0.2, ind1_Ne_0.3)
```
Summary for each ratio:

```{r}
summary(ind1_alt_ratios$ind1_Ne_0.1)
summary(ind1_alt_ratios$ind1_Ne_0.2)
summary(ind1_alt_ratios$ind1_Ne_0.3)
```


Same data in tidy format for plots and other data summaries:
```{r}
ind1_alt_ratios_long<-pivot_longer(ind1_alt_ratios, 
                                   cols = ind1_Ne_0.1:ind1_Ne_0.3,
                                   names_to="NeNc_ratio",
                                   values_to="ind1_value")
ind1_alt_ratios_long %>% select(taxonomic_group, taxon, NeNc_ratio, ind1_value)

```
Histogram faceting:
```{r}
#nicer labels
ind1_alt_ratios_long$NeNc_ratio<-as.factor(ind1_alt_ratios_long$NeNc_ratio)
levels(ind1_alt_ratios_long$NeNc_ratio)<-c("0.1", "0.2", "0.3")

hist_ratios <- ind1_alt_ratios_long %>% ggplot(aes(x=ind1_value, fill=NeNc_ratio)) +
                        geom_histogram(position = 'identity') + 
                        facet_wrap("NeNc_ratio") +
                        theme_light() + xlab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
                       theme(panel.border = element_blank(), legend.position="none")
hist_ratios
```

Boxplots:
```{r}
boxplot_ratios<-ind1_alt_ratios_long %>% ggplot(aes(x=NeNc_ratio, y=ind1_value, fill=NeNc_ratio)) +
                geom_boxplot() + 
                geom_jitter(size=.5, width = 0.1) +
                ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
                xlab("Ne/Nc ratio") +
                theme_light() + theme(legend.position="none")

boxplot_ratios
  
```

#### Supplementary Figure S8: Hist and boxplot with different Ne_Nc ratios

```{r}
plot_grid(hist_ratios + ggtitle(""),
          boxplot_ratios + ggtitle(""), ncol = 1, labels = c("A)", "B)"))

ggsave("FigS8.pdf", width = 18, height = 22  , units = "cm")
```


Summary indicator values:
```{r}
x<-ind1_alt_ratios_long %>% group_by(NeNc_ratio) %>%
                         summarize(n=n(),
                         mean=mean(ind1_value),
                         median=median(ind1_value),
                         per.0=sum(ind1_value==0) / n *100,
                         per.below.25=sum(ind1_value<0.25) / n *100,
                         per.below.90=sum(ind1_value<0.90) / n *100,
                         per.above.75=sum(ind1_value>0.75)/ n *100,
                         per1=sum(ind1_value==1) / n *100)
kable(x, digits = 2)
```

Does the value of ind1 varies with the NeNc ratio used?
```{r}
m.ratios<-glmmTMB(ind1_value ~ NeNc_ratio, family = "ordbeta", data = ind1_alt_ratios_long)
summary(m.ratios)
```



### Table for the taxa for which we have a different Ne/Nc ratio.

Select taxa for which we have an alternative known NeNc ratio:

```{r}
sp_known_ratios<-metadata %>% filter(!is.na(ratio_species_related)) %>% select(taxon, common_name, taxonomic_group, country_assessment, ratio_species_related)
select(sp_known_ratios, taxon, ratio_species_related, country_assessment)
```
Remove the elephant, because Jess confirmed <0.1 is 0.1, so no change:
```{r}
sp_known_ratios<-filter(sp_known_ratios, taxon != "Loxodonta africana")
```

Create a new variable with only numeric values for the ratio. For this, when ranges were provided we take the average.

```{r}
sp_known_ratios <-sp_known_ratios %>% mutate(
                    known_ratio=case_when(
                    # If ratio_species_related is a single value (with a decimal point), keep it as it is
                    grepl("^\\d+\\.?\\d*$", ratio_species_related) ~ as.numeric(ratio_species_related),
                    # manually change data that includes ranges or other formats. One by one:
                    ratio_species_related == "0.26 (0.1-0.3)" ~ 0.26,
                    ratio_species_related == "1:5" ~ 0.2,
                    ratio_species_related == "0.09-0.604" ~ mean(c(0.09,0.604)),
                    ratio_species_related == "0.36 (0.275-0.424)" ~ 0.36,
                    ratio_species_related == "0.1275-0.237" ~ mean(c(0.1275,0.237)),
                    ratio_species_related == "0.16-0.78" ~ mean(c(0.16,0.78))
          ))

select(sp_known_ratios, ratio_species_related, known_ratio)
```

Add English names:
```{r}
# keep only english names
sp_known_ratios<-sp_known_ratios %>% 
mutate(english_name = str_extract_all(common_name, "(?<=;|^)\\s*([^;(]+)\\s*\\(EN\\)") %>% 
                         sapply(function(x) ifelse(length(x) > 0, x, NA_character_)))
```

Show nice table
```{r}
x<-select(sp_known_ratios, taxon, taxonomic_group, english_name, country_assessment, ratio_species_related, known_ratio)
kable(x)
```


Quick summary of the new ranges:
```{r}
summary(sp_known_ratios$known_ratio)
nrow(sp_known_ratios)
table(sp_known_ratios$taxonomic_group)
```

Of the taxa for which we have a known Ne/Nc ratio, which have NcPoint data? (ie are present in `data_for_alternative_ratio`). Keep only those.

```{r}
#list taxa with known ratios
unique(sp_known_ratios$taxon)

# are they in data_for_alternative_ratio?
unique(sp_known_ratios$taxon) %in% unique(data_for_alternative_ratio$taxon)

```

Keep only the taxa with known alternative ratios and NcPoint data:
```{r}
sp_known_ratios_NcPoint<-sp_known_ratios[sp_known_ratios$taxon %in% unique(data_for_alternative_ratio$taxon), ]

x<-select(sp_known_ratios_NcPoint, taxon, taxonomic_group, english_name, country_assessment, ratio_species_related, known_ratio)
kable(x)
```

Use NcPoint data from data_for_alternative_ratio witht he known_ratio for each species to transform the NcPoint data to Ne:

```{r}
# to store loop output
data_for_table_altratio<-data.frame()

## loop through taxa with Ne kwnon ratio and NcPoint data

for(i in 1:length(unique(sp_known_ratios_NcPoint$taxon))){
# get desired taxon
desired_taxon <- unique(sp_known_ratios_NcPoint$taxon)[i]

# get known ratio for desired taxon
known_ratio <- sp_known_ratios_NcPoint[sp_known_ratios_NcPoint$taxon== desired_taxon, "known_ratio"]

# for each pop of desired taxon, estimate Ne from the known ratio
x<-filter(data_for_alternative_ratio, taxon == desired_taxon) %>%
                              mutate(Ne_known_ratio = NcPoint*known_ratio,
                                     known_ratio = known_ratio[1])

data_for_table_altratio<-rbind(data_for_table_altratio, x)
}

# check
data_for_table_altratio %>% select(taxon, NcPoint, Ne_0.1, Ne_0.2, Ne_0.3, known_ratio, Ne_known_ratio)

```
Estimate the indicator values for the 4 Ne options (0.1, 0.2, 0.3, known):
```{r}
ind1_tablesp_ratios<-data_for_table_altratio %>%
                  group_by(X_uuid, taxonomic_group, taxon, known_ratio) %>%
                  summarise(n_pops=n(),
                            n_pops_Ne_data=sum(!is.na(NcPoint)),
                            n_more_500_Ne_0.1=sum(Ne_0.1>500, na.rm=TRUE),
                            n_more_500_Ne_0.2=sum(Ne_0.2>500, na.rm=TRUE),
                            n_more_500_Ne_0.3=sum(Ne_0.3>500, na.rm=TRUE),
                            n_more_500_Ne_known_ratio=sum(Ne_known_ratio>500, na.rm=TRUE),
                            ind1_Ne_0.1=n_more_500_Ne_0.1/n_pops_Ne_data,
                            ind1_Ne_0.2=n_more_500_Ne_0.2/n_pops_Ne_data,
                            ind1_Ne_0.3=n_more_500_Ne_0.3/n_pops_Ne_data,
                            ind1_Ne_known_ratio=n_more_500_Ne_known_ratio/n_pops_Ne_data) %>%
                            ungroup()

kable(select(ind1_tablesp_ratios, -starts_with("n_more"), 
-X_uuid), digits = 2)
```

Long format for plots:
```{r}
ind1_tablesp_long<-pivot_longer(ind1_tablesp_ratios, 
                                   cols = c(ind1_Ne_0.1:ind1_Ne_0.3,ind1_Ne_known_ratio),
                                   names_to="NeNc_ratio",
                                   values_to="ind1_value")
```

Plot
```{r}
ind1_tablesp_long %>% 
  # Keep rows with different values in indicator1 within each taxon group
  group_by(taxon) %>%
  filter(n_distinct(indicator1) > 1) %>%
  # plot
  ggplot(aes(x=taxon, y=ind1_value)) +
          geom_line(colour="darkgrey") + 
          geom_point(aes(color=NeNc_ratio, shape=NeNc_ratio), alpha=0.6, size=2) +
  xlab("") + ylab("Proportion of populations within species with Ne>500 \n(Ne 500 indicator)") +
  ylim(0, 1)+
  coord_flip() +
  theme_light() + 
  theme(axis.text.y = element_text(face = "italic"),  panel.border = element_blank(), legend.position="right", text= element_text(size=13))
```



## Session Info for reproducibility purposes:

```{r}
sessionInfo()
```

