Exploring elucidation of red dye mixtures on woolen historical textiles via non-destructive spectroscopic analysis and multivariate cluster analysis

Abstract

One of the foremost challenges facing analysis of historical textiles is that the gold standard technique—high performance liquid chromatography (HPLC)—is inherently destructive. This is especially problematic considering many historical textiles are exceptionally fragile, be it from age, poor care over time, etc. One proposed solution to this is the implementation of non-destructive, namely spectroscopic, techniques, such as diffuse reflectance (Fiber Optic Reflectance Spectroscopy, FORS). In this work, 204 well-provenanced red Norwich textiles were measured with FORS and analyzed to attempt to determine chromophore combinations used to dye the original textiles. To these ends, cluster analysis algorithms and spectroscopic domain knowledge were coupled with selective HPLC validation to assess overall ability of FORS to discern changes in chromophore combinations. It was found that the near infrared (NIR) region of the spectrum contained little meaningful information in multivariate space, while the VIS region, particularly 380–469 nm, showed a narrow visible region that was primarily responsible for clustering behavior that correlates with HPLC-validated samples. This indicates that FORS shows promise for discerning chromophores in textile swatches. Additionally, X-ray fluorescence (XRF) analysis was used to confirm that the observed FORS spectral inflection point shift in the 600 nm region did not correlate with the presence or type of mordant used when dying these textiles. From this work, three main conclusions can be drawn: (1) FORS adequately identifies visual infon, which shows reasonable correlation to HPLC-validated dye recipes, warranting further investigation, and indicating utility for cois or use for those with visual impairments; (2) XRF analysis confirms that the ~ 600 nm inflection point shift and mordant are not correlated when measuring dyed textiles, unless mordant is present below analytical detection limits or not present at all; (3) many documented structural-to-spectral relationships established in the conservation literature are too weak in dyed textiles for statistical analysis and, by extension, expert spectral identification.

Introduction

The scientific study of tangible cultural heritage/material culture provides critical information to understand human history and how technologies have evolved; while non-destructive identification of constituent materials (e.g. dyes on textiles) helps with their preservation. It is good scientific practice in conservation science to only sample when it is essential and possible, due to the unique and fragile nature of objects of cultural significance [1].

High performance liquid chromatography (HPLC) with spectroscopic or mass spectrometric detection is a micro-destructive analysis technique widely accepted as the most reliable option for dye identification, even when the characteristic chemical markers or chromophores are degraded, and/or present in nanogram amounts [2]. Because it is unfeasible to use HPLC to analyze all objects, researchers have explored alternatives such as gas chromatography (GC) [3, 4], surface enhanced Raman spectroscopy (SERS) [5,6,7,8], and fiber optic reflectance spectroscopy (FORS) [8,9,10,11,12,13,14,15]. Of the aforementioned techniques, only FORS is non-destructive, and it can even be non-invasive, using a non-contact probe, making it desirable for analysis of cultural objects. FORS detects electronic transitions from the ultraviolet to the visible range, as well as vibrational overtones in the infrared region, making it suitable for color compounds like the chromophores present in dyes [16].

The challenges of dye analysis span from lack of detailed historical sources that point to plant references, as it is the case with South American plants (e.g., Cosmos sulphureus),[17] to the numerous dye combinations that may be present in a historical textile. Using historical swatches to create reference datasets is ideal, especially when it is known that they are composed of defined combinations of chemical compounds, as in the case of Norwich textiles [18]. FORS is reported to be a powerful examination tool to identify anthraquinone dyes and some mixtures in both freshly colored, and artificially degraded textiles. However, the modest chemical specificity of the technique narrows its potential for trace analysis problems like analyzing dyed fabrics.

FORS presents cost and portability advantages, but its wavelength resolution is lower than other better-established techniques like Fourier transform infrared spectroscopy (FTIR) [12]. Vibrational information from specific dyes is difficult to discern on textiles because the characteristic chromophores are near their detection limits (nanograms) and the contribution of the matrix is preponderant [2]. The successes with dyes on textiles and lake pigments reported by Angelini, et al. [10], Vitorino et al. [19] and Maynez-Rojas et al [20]. show that this technique is feasible for dye identification. In practice, stakeholders interpret FORS or reflectance data by comparing their results to references, thus, a way to increase FORS’s accessibility includes making more spectra available, perhaps as databases, e.g., Montagner, et al. library of modern dyes obtained from pattern cards [21]. However, it is unrealistic to have a database that contains diffuse reflectance/FORS spectra for every natural dyestuff as well as their combinations in a variety of fibers at different degradation stages—at least for now. In lieu of a comprehensive database, an alternative is presented relating reflectance spectral data connected to the chromophores identified by HPLC in naturally aged, dyed fibers of Norwich textiles (1783–1831). The long-term aim is to extend the methodology using FORS and chemometrics to other types of textiles dyed with the identified chromophores.

A library, database, or methodology built with Norwich textiles has great potential given that (woolen) textiles dyed with natural dyes are common in historical collections worldwide. Norwich, UK, was a pre-eminent textile producer from the 16th to the second half of the eighteenth centuries. Its origins trace back to 1565, when Dutch and Walloon immigrants arrived to a textile-expert city [22]. Norwich’s textile production resembled “factories in miniature” and included dedicated workers that specialized in tasks like dyeing. In Norwich’s most productive period, dyers were divided into competitive dyeing houses that protected their dye recipes just like patents protect industrial processes today [22]. This is why historians speculated about existing individual recipes attributed to specific dyeing houses, i.e., unique combinations of chromophores present in each hue, associated to a specific dyeing house. Continued work on Norwich fabrics strengthens historians’ hypothesis: after analyzing ca. 100 textiles (using XRF, FORS, and in selected cases HPLC), including those discussed in this paper, well-defined combinations of chromophores were consistently found, which seems indicative of pre-established dye combinations or recipes [18].

Similar to today’s manufacturers using sales catalogues, the people selling these goods between the late 18th and early nineteenth century used patternbooks. These objects are invaluable historical records: each dye house made their own, using textile swatches organized by “pattern” or number, and used them as sales catalogues (Fig. 1). Patternbooks are hand-bound books with unused, naturally aged, textile swatches, whose analysis provides different information than that obtained from swatches recently dyed and/or subjected to accelerated degradation [20, 23], and likely more similar to what stakeholders would encounter.

Fig. 1

Two pages of a Norwich patternbook produced by John Christopher Hampp (1750–1825) in Norwich, UK. Patternbooks contain textiles organized by number or pattern. Textiles 1027–1030 were analyzed in this manuscript. Courtesy, Winterthur Museum, Garden, and Library. Joseph Downs Collection. Accession Number 65 × 695.6. Photo credit: J. Schneck

The dyes and the characteristic chromophores present in these textiles or patterns are naturally sourced, and are likely present in other textiles of similar age. Therefore, developing a non-destructive approach to studying Norwich textiles is expected to be extended to understanding fabrics of similar substrate and age; and in turn, to extending the approach to other textiles and dyed substrates (e.g., paper). Typical use of FORS in the literature identifies dyes in textiles via manual interpretation [20, 24,25,26,27,28,29,30], with only a handful of works employing multivariate analyses [31,32,33,34]. The present work builds on the limited body of multivariate analysis of FORS spectra by first, significantly increasing dataset size; second, expanding analysis to identifying spectral regions that correspond to multivariate clustering; and lastly, validating these findings with selected HPLC–MS analyses. Through the application of chemometrics and multivariate cluster analysis algorithms, this paper extends the early understanding of information contained within non-destructive FORS spectra of dyed historical textiles and relates this information to chromophores identified by HPLC.

Experimental

Samples and sampling

The term “sample” will be used only when a physical sample was taken from an object for micro-destructive analysis. Swatch will be used to refer to a given textile in its entirety from which measurements were taken; and the term “measurement” will be used for FORS and XRF spectra or HPLC chromatograms collected from individual textile swatches.

This study consisted of fabric swatches from four Norwich patternbooks housed in the Joseph Downs Collection of Manuscripts and Printed Ephemera, part of the Winterthur Museum, Garden, and Library, following best-practice guidelines—an example from one of the four books is shown in Fig. 1.[1, 35] Across the four books, there were 204 textile swatches, which were measured by FORS (25 spectra/swatch), XRF (5 spectra/swatch), and HPLC (1 chromatogram/selected swatch), as described below in "Fiber Optic Reflectance Spectroscopy (FORS) Collection", "X-ray Fluorescence (XRF) Collection", and "High Performance Liquid Chromatography" sections respectively. For HPLC analysis, 20 samples (10% of the dataset) were selectively chosen for dye analysis. This decision was made after weighing the advantages and disadvantages of using destructive analysis to (a) provide scientifically sound information on Norwich’s textile practices and trade; and to (b) use the gathered information to develop an alternative to non-destructively study textile collections worldwide.

Glazed and unglazed swatches are approximately 1 × 15 cm, and they are adhered to a paper page using animal glue [18]. Spectral variation across glazed and unglazed swatches was deemed negligible. Of note, earlier work suggested that the glazes were natural gums.

All raw data and analysis code can be found in the Additional file 1.

Fiber optic reflectance spectroscopy (FORS) collection

(Diffuse) reflectance spectra were collected using an ASD Inc. FieldSpec 4 Hi-Res portable FORS spectrometer equipped with a 512 Silicon array (visible near infrared, VNIR), and two thermoelectrically cooled fast scan InGaAs photodiodes (shortwave infrared spectroscopy, SWIRS), covering the wavelength ranges $\le 1000$ nm, $1000<\lambda \le 1451$ nm, and $>1451$ nm. A bifurcated reflectance probe (~ 2 mm diameter, Malvern Panalytical Inc.) consisted of a common end of six fibers (600 $\upmu$ m diameter). The sampling configurations tested are shown in Fig. 2, and included offsetting the illumination source and the detection probe (Fig. 2a) [36]; illumination and detection perpendicular to the swatch (Fig. 2b); and illumination and detection perpendicular to the swatch but held in a pistol grip (Fig. 2c).

Fig. 2

Configurations tested for fiber optic reflectance spectroscopy tested. a Detection probe and external light source angled at the same swatch spot; b bifurcated probe mounted on a metal holder at a 90° angle from the textile without external illumination; and c bifurcated probe mounted on a pistol grip

The chosen configuration is shown in Fig. 2c, as this allowed greater sampling precision and most importantly, reproducibility. In this configuration the fiber is held approximately 2 mm above the grip’s flat surface, resulting in increased signal intensity. The instrument was calibrated using a white reference Spectralon® panel. Spectral processing within ViewSpec Pro v. 6.2.0 (ASD Inc.) was used solely for splice correction between the three detectors; all other spectral processing was done using R 4.1.0, as is detailed in "Chemometric Analysis" section.

Five spectra were collected at five different locations of each swatch in a right to left sampling direction spacing measurements approximately 1 cm, for a total of 25 spectra per swatch. Each spectrum had a measurement time of 10 s. To ensure spectral consistency throughout the data, a subset of the collected data was analyzed for spectral variance at each spot, and across each swatch. There was minimal non-random variation both within each spot and between spots on the same swatch, indicating that all spectra from a given swatch could be averaged into a single measurement to improve signal-to-noise, if necessary. Both glazed and unglazed swatches displayed the same behavior.

X-ray fluorescence (XRF) collection

Non-destructive, qualitative, energy-dispersive (ED) XRF spectra were collected using a Bruker ARTAX μXRF spectrometer using a rhodium tube (600 μA current, 50 kV voltage, 100 s live time irradiation, approximately 70–100 μm spot size) with an element detection range of potassium (K) to uranium (U). For each spectrum, a built-in video camera and laser were used to focus the spectrometer on the swatch. Each fabric swatch was measured at five unique spots with one measurement taken per spot, yielding a total of five spectra per swatch. The paper used in each patternbook was used as a reference. Sampling locations were intentionally randomized on each swatch. Five XRF spectra were deemed sufficient to be directly compared with the five spectra at five locations collected for FORS due to the minimal single-spot variation, as explained in 2.2. Ar, Rh and As were present in all textiles: Ar from air is expected to be detected because no vacuum was used; Rh from the X-ray source tube; and As is possibly connected to fumigation. Previous historical research suggested that alum, copper/iron sulfate, tin chloride, and arsenic were used as mordants. But since As was detected in every page analyzed, it is more likely indicative of an earlier fumigation, coinciding with observations in previous work.¹⁷ The other identified elements were: S, Sn, Ca, Ti, Fe, and Cu.

High performance liquid chromatography

HPLC analysis via diode array detector (HPLC–DAD) and liquid chromatography—electrospray ionization mass spectrometry quadrupole time of flight (LC-ESI-Q-ToF)

High performance liquid chromatography was used to identify chromophores present in textiles, and relate them to a specific dye combination (recipe) of a subset of the dataset. An average of 4 mm of a single thread was taken from each of the textile swatches. HPLC–DAD analyses were performed on a Jasco International Co. (Japan) system consisting of a PU-2089 quaternary pump equipped with a degasser, an AS-950 autosampler, and an MD-2010 spectrophotometric diode array detector. ChromNav (Jasco International) software was used for data acquisition and processing. The diode array detector (DAD) acquisition was performed in the range of 200–650 nm every 0.8 s with 4 nm resolution.

The HPLC-ESI-Q-ToF system consisted of a HPLC 1200 Infinity, coupled with a quadrupole-time of flight mass spectrometer Infinity Q-ToF 6530 detector by a Jet Stream ESI interface (Agilent Technologies). The ESI conditions were: drying and sheath gas N₂, purity 98%, temperature 350 °C, flow 10 L/min and temperature 375 °C, flow 11 L/min, respectively; capillary voltage 4.5 kV; nebulizer gas pressure 35 psi. The fragmentor voltage was 175 V; nozzle, skimmer and octapole RF voltages were set at 1000 V, 65 V and 750 V, respectively. The high-resolution MS and MS/MS acquisition range was set from 100 to 1000 m/z in negative mode, with 1.04 spectra/sec acquisition rate. For the MS/MS experiments, 30 V were applied in the collision cell (collision gas N₂, purity 99.999%). The quadrupole mass bandpass used during MS/MS precursor isolation was 4 m/z. Agilent tuning mix HP0321 was used daily to calibrate the mass axis. MassHunter® Workstation Software (B.07.00) was used to carry out mass spectrometer control, data acquisition, and data analysis.

For both the systems, the chromatographic separation was performed at 30 °C on an analytical reversed-phase column Poroshell 120 EC-C18 (3.0 × 75 mm, particle size 2.7 μm) with a pre-column Zorbax (4.6 × 12.5 mm, particle size 5 μm), both Agilent Technologies (Palo Alto, CA, USA). Separation used water and acetonitrile (ACN) HPLC grade (both Sigma Aldrich, USA), both modified with 0.1% (v/v) formic acid (FA, 98% purity, J.T. Baker, USA). The flow rate was 0.4 mL/min and the program was: 15% B (0.1% FA in ACN) for 2.6 min, then to 50% B in 13.0 min, to 70% B in 5.2 min, to 100% B in 0.5 min and then held for 6.7 min. Re-equilibration took 11 min. Eluents for HPLC-ESI-Q-ToF analyses were water and acetonitrile, both LC–MS grade (Sigma-Aldrich, USA).

These identifications were then taken as verification of dye recipes and used for analysis in "High Performance Liquid Chromatography (HPLC) Verification" section.

Sample pre-treatment

Chromophores were extracted from threads using a mild pre-treatment with chemicals used as received without further purification. After adding 200 µL of a 0.1% Na₂EDTA (Fluka, USA) in H₂O/ N,N-Dimethylformaldehyde (DMF) (1:1, v/v) (DMF, 99.8% purity, J.T. Baker, USA) solution to the sample, samples were sonicated in an ultrasonic bath at 60 °C for 1 h. Extracts were filtered using Polytetrafluoroethylene (PTFE) filters (0.45 µm pore size). 15 µL of each extract were injected in both the HPLC–DAD and HPLC-Q-ToF systems.

Chemometric analysis

All data analysis was done in Rstudio version 1.4.1717[37] running base R version 4.1.0 [38], and using the packages ‘signal,’[39] ‘rgl,’[40] ‘stats,’[38] and ‘mclust.’[41] Data were imported as.csv files and combined into a single data matrix in Rstudio. Data matrices had ${\text{m}}\times {\text{n}}$ dimensions where rows (m) are spectra and columns (n) are swatch intensities at measured wavelengths and energies for FORS and XRF, respectively. All data and code used to analyze said data discussed in this manuscript can be found in Additional file 1. To encourage reproducible science, see the Supporting Information for access to the full code and dataset used for this study.

Preprocessing FORS

The splice corrected FORS dataset was imported into Rstudio. The raw spectra covered the range 350 nm to 2500 nm with 1 nm spectral resolution $\left(\mathrm{\Delta \lambda }\right)$ prior to trimming the first 30 wavelengths, as they were unusable due to substantial noise. After trimming, the spectral range began at 380 nm with the tail end remaining unchanged. Spectra associated with six outlier swatches were subsequently removed. The trimmed-data were baseline corrected (row-wise) by subtracting the minimum value of each spectrum from each point of the respective spectrum (Eq. 1) to reduce measurement-to-measurement intensity variation.

$${{\varvec{X}}}_{baseline}={{\varvec{x}}}_{ij}-{\text{min}}\left({{\varvec{x}}}_{i}\right)$$

(1)

Following baseline correction, data were smoothed, and baseline offset was removed by using a Savitzky-Golay first derivative smoothing filter using a second order polynomial, first order derivative, and a 9-point window (n = 9, p = 2, m = 1). These parameters were chosen because: (a) a 9-point window was observed to minimize spectral distortion because the number of points in a given peak were over twice the filter window length; (b) a second order polynomial was better than higher order polynomials at suppressing random noise; and (c) a first derivative was observed to remove baseline offsets while minimizing the amount of noise added to spectra—higher order derivatives increase baseline noise, so are reserved for sloping or curved baseline removal.

Data were then trimmed to lengths corresponding to spectral features of interest that will be specified when each analysis is discussed. All data were autoscaled (z-score scaled) prior to analysis. All subsets of preprocessing shown in Figs. 5.6 and 5.7 were autoscaled prior to analysis. Raw and preprocessed data are shown in Fig. 3a–c while removed outlier swatches discussed above are shown in Fig. 3d.

Fig. 3

Full dataset consisting of 204 swatches for a raw spectra, b baseline corrected spectra, c preprocessed spectra, and d spectra of the 6 outlier swatches removed from the dataset. For a and b, reflectance is given in percent expressed as a decimal

Preprocessing XRF

The raw XRF data were imported prior to removing leading and trailing zeros in the spectra. Preprocessing varied by analysis and included (a) Savitzky-Golay smoothing using a second order polynomial, first order derivative, and a 15-point window (n = 15, p = 2, m = 1), (b) manual variable selection to remove all non-elemental peaks, (c) a combination of both (a) then (b). The rationale for each is provided in Results and Discussion. Raw and preprocessed data are shown in Fig. 4.

Fig. 4

XRF spectra of 204 swatches a raw and b preprocessed. The main XRF lines correspond to K⍺ of S, Sn, Ca, Ti, Fe, Cu, and As

Algorithms

Since the data classes were unknown, data analysis is inherently limited to exploratory techniques and clustering algorithms. Therefore, analyses included Principal Component Analysis (PCA); decision rules/classifiers manually chosen based on noteworthy spectral features; multiple least squares regression (MLSR); model-based clustering (MBC); and k-means clustering.

PCA is a common unsupervised exploratory data analysis algorithm that enables understanding of the largest directions of variance within an N-dimensional dataset [42]. The largest directions of variance are expressed as linear combinations of the variable space which are then projected to a lower dimensional space (typically 2D or 3D) for visualization, with the underlying assumption that swatches that are more similar in N-dimensional space will cluster in reduced-dimensional principal component space. For this reason, PCA is a good tool to assess if a given classification or cluster analysis problem is worth pursuing.

Similar to PCA, the cluster analysis algorithms k-means clustering and model-based clustering find clusters in the original data as well. The difference between PCA and k-means is that the latter is a distance based algorithm which requires a user-specified number of clusters, k [43]. K-means works by randomly choosing k points from the original data set, calculating a distance metric, then assigning points to each of the k clusters based on which center they are closest to. The “centers” are then moved to the center of each of their respective clusters and the distance is recomputed. The distance calculation and cluster assignments are then iteratively repeated until the algorithm converges.

MBC is a more complex extension of k-means clustering consisting of an iterative process that assumes the data is created from a finite combination of models [44]. It classifies the swatches by assuming k models/classes and subsequently determines from which of the k models the swatch in question came.

Multiple Least Squares Regression (MLSR) is an extension of ordinary least squares regression that determines the contributions of vectors of x-variables in a matrix to multivariate y-variable(s). This is in contrast to ordinary least squares regression (OLS) which determines the contributions of a single vector of x-variables to a single vector of corresponding y-variables, i.e., observation of how a single parameter, y, changes with respect to a changing x-variable.

Lastly, decision rules are classifiers created with the use of some attribute of the spectral data to create pseudo-classes or clusters within the data. Decision rules will be discussed in greater detail below.

Reflectance algorithms

FORS data were preprocessed and analyzed using PCA for both exploratory analysis and cluster analysis, which were performed on both full spectra and three selected spectral regions, chosen because of their high variance: the entire visible region, the entire near-infrared region, and selected subsets of visible and infrared regions. Only a subset of these analyses are presented for brevity.

The spectral variation observed in the visible portion of the spectra—specifically 399–469 nm—led to creating a decision rule classifier (boundary condition) that grouped the data in 3D principal component (PC) space. The decision rule classifier was created by first finding the maximum intensity peak’s wavelength in the 399–469 nm range; followed by plotting frequency versus maximum wavelength in a histogram (Fig. 5). Data were divided into three clusters, establishing boundaries at density minima, using the same color coding as other figures for clarity: (1) spectra with no peak in this region (max $\le$ 384.5 nm) in red, (2) spectra with a peak between 384.5 nm and 405 nm (384.5 nm < max < 405 nm) in blue, and (3) spectra with a peak greater than or equal to 405 nm (max $\ge$ 405 nm) in green.

Fig. 5

Density plot showing the predominant wavelengths at which peak maxima were observed in the 399–469 nm spectral region. Vertical purple lines indicate where decision rule boundaries were chosen

In addition to creating a peak-intensity-based decision rule, clusters observed in PC space were verified with MBC and k-means Clustering. The mclust algorithm was applied to trimmed, baseline corrected, Savitzky-Golay smoothed data, which was also mean-centered and variance-scaled (z-scaled) within the same algorithm. Clusters were manually set to two, three, four, and five clusters, with three clusters showing optimal results. This process was repeated with the Hartigan-Wong k-means algorithm using 1000 starts as well as two, three, four, and five clusters on the PCA scores, with three clusters showing optimal results again. Comparison of MBC-based classes to intensity-based-decision-rule assigned classes enabled assessment of decision rule efficiency. MBC classes were treated as “true” while decision rule classes were treated as “predictions” so that the percentage of each swatch assigned to a “true” class could be calculated (Table 2). This process was repeated for k-means-assigned classes (Table 3). Tables 2 and 3 are confusion matrices for each respective classification task, where rows represent the “true” cluster-analysis-determined value of each cluster and columns represent the predicted value of each cluster.

XRF algorithms

The preprocessed XRF data were also analyzed with PCA, as with FORS data. Additionally, XRF data underwent MLSR analysis with the PC2 scores vector of the 579–629 nm spectral region used at the y-block, explained in detail in the "Results and discussion" section.

Results and discussion

Along with historical context and connoisseurship, chemical information of cultural heritage objects obtained through instrumental analysis is the gold standard to determine objects’ provenance. This chemical information is also key to deciding on best ways of storing, treating, and exhibiting precious objects; and as objects are unique, sampling-based techniques are the last option, assuming stakeholders can access them. Liquid chromatographic techniques remain the most precise to analyze dyed textiles, with some instruments needing less than 1 mg/0.3 mm single thread to provide even quantitative information. But best-practice guidelines still favor non-sampling techniques like FORS, whose published success inspired the strategy discussed here [20, 24, 26, 34]: attempting the creation of databases that provide spectral references using FORS, each of which has been validated using HPLC. To minimize the problems associated with using newly dyed swatches, the references used were well-provenanced, naturally aged, historical swatches from Norwich, UK. These swatches were chosen because previous studies had suggested that historical manufacturers used a limited number of dye combinations, with cochineal and madder prevailing on red swatches [18].

Fiber optic reflectance spectroscopy (FORS)

Near-infrared (NIR) Region

Data analysis began by applying PCA to the full, preprocessed FORS dataset with minimal success (Fig. 6a), as expected given the limited number of spectral bands corresponding to analytes of interest (i.e., chromophores) relative to the total number of spectral channels. Principal components (PCs) 1–3 (31.82% total variance, Table 1) and PCs 1–10 (56.93% total variance, Table 1) showed similar results in 3D PC space: no clustering, minimal zoning, and no real discernable directions for classifying dye or chromophore combination profiles. Plausible causes include: (1) that the useful information is overwhelmed in the PCA by uninformative variables; (2) absence of classification of useful information present in this dataset; or (3) information being below the instrumental detection limits. Since (1) is common when PCA is applied to a full spectral data space, it is necessary to eliminate noisy channels and other low variance variables that prevent PCA from finding chemically informative spectral regions, e.g. cochineal band (~ 423 nm) [20].

Fig. 6

PCA scores plots of a the full spectral region 380–2500 nm, b the full NIR region 760–2500 nm, c the high variance 1230–1500 nm subset of the NIR region, d the high variance 1780–1900 nm subset of the NIR region, e the high variance 1950–2050 nm subset of the NIR region, and f the combination of high variance regions shown in (c–e). The different scales are intentional to ease visualization. Color indicates individual swatches

Table 1 Summary of PCA Cumulative Variances for all Spectral Subsets across the full spectral region (380–2500 nm); PCs 1–3 and PCs 1–10

The dataset was divided into the visible (380–759 nm, VIS) and near-infrared (760–2500 nm, NIR) regions. The preprocessed data showed minimal variance within the near-infrared region except for a few wavelength ranges (1230–1500 nm, 1780–1900 nm, and 1950–2050 nm). Reapplying PCA to the full 760–2500 nm NIR region (Fig. 6b) showed near-identical results to the full dataset across the first 3 PCs and first 10 PCs which captured 32.94 and 54.44% total variance, respectively (Table 1).

There were three regions that showed noticeable variation, and they were isolated to minimize the effect of uninformative variables overwhelming the final PCA model: 1230–1500 nm (Fig. 6c), 1780–1900 nm (Fig. 6d), and 1950–2050 nm (Fig. 6e).

The overall observations for higher variance regions remain the same as in the IR region, except for a few outliers. The first 3 PCs show total variances of 68.80%, 52.32%, & 78.42% while the first 10 PCs show total variances of 83.18%, 86.00%, & 95.97% for Fig. 6c–e, respectively (Table 1). One notable exception is that PCs 2 and 3 show six-near outliers, while the remaining 192 swatches lie within a single cluster (Fig. 6d). Since minimal information was extracted from these three regions, combining the spectral regions associated with Fig. 6c–e was also uninformative (Fig. 6f)—with the first 3 PCs accounting for 50.21% total variance and the first 10 PCs accounting for 75.88% total variance (Table 1).

Figure 6b–f indicates that the variance in the NIR region is not associated to useful vibrational information for classifying textiles based on chromophore/dye combination, and was therefore omitted from the remaining analyses. The results demonstrated that current state-of-the-art FORS lacks chemical specificity in the IR region: chromophores used to dye textiles are in the microgram scale [29], making this a trace analysis problem that requires more selective instrumentation with long collection times. Current literature suggests that FORS can identify some dyestuffs using reflectance information obtained between 300 and 1000 nm, both confirming that the 760—2500 nm region analyzed here would show minimal clustering in PC space and motivating the next portion of this study, which is the investigation of the VIS spectral region (380–759 nm) [6, 20, 24, 26, 29, 45,46,47,48,49].

Visible (VIS) region and decision rule

Like the NIR region, the VIS region was divided into visually high variance subsets. PCA was then applied to the full VIS region as well as the selected subsets. Figure 7a shows PCA of the VIS spectral region 380–759 nm where two definitive—perhaps three, albeit not as clear—clusters could be distinguished, suggesting FORS can discern swatches based on visible information.

Fig. 7

PCA scores plots of a the VIS spectral region 380–759 nm, b the VIS spectral region recolored based on the peak maximum decision rule—points that correspond to swatches with HPLC results are replaced with photographic images of those swatches to assess color variation across PC space, c the 380–469 nm VIS region used to create decision rule d the 510–555 nm subset of the VIS region associated with cochineal and madder, e the combination of data from c, d, and f the 579–629 nm region associated with the FORS inflection point. The different scales are intentional to ease visualization. The color in a indicates individual swatches while b–f is an overlay of the clusters determined through the decision rule created in Fig. 5

Because distinguishing between the presence of two or three clusters was ambiguous, a small subset of spectra was taken from each of the possible three clusters and inspected further. The smaller cluster defined roughly between -30 and 10 on PC 1 and between -30 and -10 on PC 2 showed no peaks in the 380–470 nm region, while the other one/two clusters were blurred because of similar peaks that were offset by a few nanometers, possibly hinting that the blurred clusters are, in fact, distinct. This observation was further strengthened by the presence of a distinct inflection point (559–659 nm). Inspection of the textile swatches of each of the subset spectra showed that clustering coincided with hue: the smaller cluster corresponds to darker reds, the large cluster corresponded to darker pinks, and the blurred cluster corresponded to lighter pinks (Fig. 7b).

The decision rule explained in the "Experimental" section was created to assist in determining whether two or three clusters were present (Fig. 5). Figure 7b shows the PCA scores plot from Fig. 7a using the colors assigned with this decision rule, overlaid on the VIS region (380–759 nm; first 3 PCs—74.28% total variance & first 10 PCs—91.83% total variance, Table 1). This decision rule indicates that the primary source of variance responsible for clustering is the presence or absence of the spectral peak(s) under 469 nm. The colors overlaid on the 380–759 nm VIS PCA corresponded to only a small portion of that region (380–470 nm), indicating that part of the VIS region contained potentially uninformative variables. Various regions of the VIS spectra were isolated to explore this hypothesis using PCA: the autoscaled preprocessed (1) 380–469 nm region used to create the decision rule (Fig. 7c); (2) the 510–555 nm spectral region cited by literature as the primary absorption region of cochineal and madder (Fig. 7d); (3) the combination of the 380–469 and 510–555 nm regions Fig. 7e; and (4) the 579–629 nm range over which the FORS inflection point occurs (Fig. 7f).

Below are preliminary conclusions that can be drawn from Fig. 7.

- The variance within the VIS region is sufficient to cluster data in 2–3 PCs, as evidenced by ~ 75% of the total variance being contained in the first 3 PCs. This indicates there is sufficient variance in this region to overcome the influence of uninformative variables (Fig. 7a, b).

- The peaks used to create the decision rule are sufficient to cluster the data into three “classes,” indicating peak max alone contains significant meaningful variance (Fig. 7c). Note that “classes” is denoted using quotation marks because in chemometrics a classification problem is defined by knowing the identities of all samples prior to statistical treatment [50]. The first 3 PCs correspond to 80.70% of the total variance, and the first 10 PCs account for 93.28% of the total variance (Table 1). This relates well to naked-eye observations, as this wavelength range corresponds to red/pink, the bulk textile color.

- The clusters observed in PCA correlate better with the VIS region used to create the decision rule than with the reported wavelength region of 510–555 nm, associated with the detection of cochineal and madder [27, 30]. Figure 7c shows relatively distinct clusters while Fig. 7d shows reduced inter-cluster variance between the red and blue clusters as well as the division of the green one into three separate clusters. Despite the 510–555 nm region showing less clustering, it captures more total variance across the first 3 PCs (93.48%) and the first 10 PCs (98.74%), shown in Table 1, indicating that nearly all information that could be used for clustering has been extracted.

- The VIS region is a better predictor than the 510–555 nm region, as assessed by the first three PCs, because the latter led to worse clustering. Both regions, however, could be suitable for classifying red textiles of similar provenance, which means that both bulk color information and slight spectral differences based on dye recipe may be a consistent classifications tool than the 510–555 nm region: they held greater scaled weight and lower cluster scatter (Fig. 7c, d).

- The above was confirmed by repeating PCA on the combination of both regions (Fig. 7c, d), which showed the Fig. 7e clusters remained about as separated as the VIS region (Fig. 7c). The spread is still greater than without the inclusion of spectral data from the 510–555 nm region (Fig. 7d), particularly of the green cluster. What is worse, the percent total variance captured decreased to 79.45% for the first 3 PCs and to 91.48% for the first 10 PCs (Table 1), further strengthening the evidence of VIS being a better predictor.

- Application of PCA to the 559–649 nm inflection point region shows it has limited discriminatory power—only along PC 1 (Fig. 7f). This accounted for 97.67% total variance for the first three PCs and for 99.86% of the total variance through the first 10 PCs (Table 1). Bonding of a chromophore to a metallic center, as is the case in a mordanting process, would lead to a band shift [47]. Suspecting the observed change in the inflection point shifts were correlated to mordants in the dyed textile, an XRF dataset was collected and treated in a similar fashion, which will be discussed in "X-Ray Fluorescence (XRF) Analysis" section.

Decision rule assessment

“Classes” generated through the decision rule were compared to the “classes” returned using the model-based clustering algorithm, as well as the k-means clustering algorithm. MBC was run on the 380–759 nm z-score scaled, preprocessed spectra specifying three clusters; while k-means was run on the full rank PCA scores from Fig. 7a (380–759 nm) also specifying three clusters. Clustering algorithms will always find clusters regardless of real observable clusters, and thus, two, three, four, and five clusters were tested for both MBC and k-means clustering. Based on these results, three was determined to be the optimal number of clusters for this problem using MBC and k-means clustering. Should the three MBC or k-means clusters overlap well with clusters created by the decision rule that used chemical information, clustering can be assumed to be primarily governed by the 380–469 nm wavelength range (Fig. 7c), i.e., clustering algorithm “classes” will be considered “true” and decision rules “classes” will be considered “predicted” allowing for the creation of a confusion matrix.

The average classification across the three classes revealed that the peak maximum from 380–759 nm corresponds to 80.32% classification success relative to MBC, and 80.54% success relative to k-means clustering. The 380–469 nm region that corresponds to Fig. 7c yields 91.31% relative to model-based clustering (Table 2) and 87.87% success relative to k-means clustering (Table 3). The 510–555 nm region from Fig. 7d unsurprisingly showed only 61.57% success for MBC, which accounts for the comparatively lower overlap, as significantly more than three “classes” were observed. The same region showed 79.44% success for k-means clustering, indicating this region is more easily clustered in principal component space relative to spectral space. The combination of both 380–469 nm and 510–555 nm regions corresponding to Fig. 7e shows 85.95% success for MBC, and 88.83% success for k-means clustering. Lastly, the 579–629 nm region associated with Fig. 7f shows 85.94% success for MBC and 66.42% success for k-means clustering.

Table 2 Confusion matrix for model-based clustering of 380–469 nm versus the decision rule clusters

Table 3 Confusion matrix for k-means clustering of 380–469 nm versus the decision rule clusters

The above results show the majority of the information (87.96 for MBC & 91.66% for k-means) used for clustering in the full VIS spectral range is present in the first 90 wavelengths (380–469 nm) used for creation of the decision rule. This indicates that only a small portion of the total variance is included in the 470–769 nm region. Further, MBC and k-means proved less efficient at defining clusters, contradicting reports that focus on the 510–555 nm region to determine presence of colorants based on spectral information beyond the initial visible peak [20, 25,26,27,28, 34, 48, 51,52,53]. This is strengthened by the percent success increment when combining the two regions (Fig. 7e), indicating that the 380–469 nm wavelength region contributes significantly more than the literature-specified 510–555 nm wavelength region.

Both the FORS PCA and classification accuracy results point to the chemical information contained in the 380–459 nm region as being the most significant source of spectral variance. However, the inflection points region (579–629 nm, Fig. 3c), may contain some spectral classifying information, as indicated by (1) the zoning of decision rule “classes” in PC space along PC 1, and (2) the higher classification success of MBC relative to k-means. Since XRF identifies elements commonly associated with historically used mordants, XRF was explored as a complementary technique. This was to explore if the inflection points observed were indeed associated with chromophores bonded to metals present in mordants, as the currently accepted model suggests.

X-ray fluorescence (XRF) analysis

Suspecting the inflection point shift may be related to existing mordants (Fig. 7f) an XRF dataset was collected on the same swatches to assess whether elemental profiles were indicative of mordant-related spectral differences and if so, if they coincided with hue differences [47]. As with the FORS data, PCA was applied to the trimmed Savitzky-Golay preprocessed XRF spectra (Fig. 8a) and preprocessed variable selected XRF data (Fig. 8b). Variable selection for the XRF data was manually done to select all elemental peaks while explicitly removing baseline. There is minimal clustering present in both the preprocessed and variable selected data. The non-variable selected data potentially shows two clusters, but this separation is removed after variable selection. Hence, it is likely that clusters resulted from overfitting through modeling noise.

Fig. 8

PCA Scores plots for the (a, c) XRF trimmed & preprocessed spectra, (b, d) XRF trimmed, preprocessed, & variable selected spectra. Points are colored by (a, b) swatch, and (c, d) FORS decision rule “classes” following the same color coding as in Fig. 7. The different scales are intentional to ease visualization

The colors associated with the three “classes” established by the decision rule were overlaid onto the XRF PCA (Fig. 8a, b) for both the preprocessed (Fig. 8c) and variable selected data (Fig. 8d). This visualization method was used to assess if the FORS inflection point shift was correlated with elemental profiles (XRF) from mordants on textiles. Due to the data size mismatch with FORS having 25 spectra/swatch and XRF having five spectra/swatch, the most frequent color assignment for each spot (statistical mode) was used for the decision rule color scheme. In both cases, this overlay shows that there is no clustering based on signal nor is there discernable zoning associated with the three classes observed in FORS. Because the PCA scores indicate the largest directions of variance are random, it is probable that the FORS inflection point is not correlated with any electronic or vibrational chemical information.

Multiple least squares regression (MLSR) was performed to further assess correlation between the FORS classes, the FORS inflection points region, and mordant elemental profile. MLSR regressed the first FORS principal component (PC 1) scores axis against the preprocessed variable selected XRF data. This vector was chosen because it is the primary axis of the peak shift, as indicated by linear separation across the parabolic shape of the data (Fig. 7f). The MLSR model would show data falling closely to the diagonal 1:1 line with “class” colors zoning along the length of the same line if the three classes observed through FORS analysis were adequately predicted based on elemental profile alone (XRF).

Figure 9a, however, shows the above is not the case. Class zoning persists only because the regressor variable already showed zoning. The data cluster has a significantly different slope than one, leaving a large portion of the “class” shown in blue below, and the green “class” almost entirely above, the prediction line. This indicates that a significant portion of blue “class” and almost all of the green “class” was predicted to be closer to the center of the values—where the “red” class is predicted. Further, the R² value (indicating what percentage of the variance in y is directly related to x) for this model is 0.481, and the adjusted R² (adjusts R² for the number of predictors in the model) is 0.1528, showing this is a poor prediction model.

Fig. 9

a Predicted vs. Measured plot for the linear model relating PC 1 of the FORS inflection point PCA (579–629 nm) to the preprocessed variable selected XRF spectra. b Residuals vs. swatch plot to assess if a linear model is appropriate for this data

To assess model linearity, the residuals were evaluated as a function of swatch number (Fig. 9b), showing homoscedastic residual distribution (constant across swatches) and a large residual standard error of 7.335, thus proving that a linear model is appropriate for this data. There is a linear relationship of the inflection points region and elemental profile, albeit that linear relationship does not correlate well to elemental composition of the swatch.

Inflection point shift in FORS is therefore not related to chromophore-mordant information, at least in the case of dyed textiles—their chromophore concentration is intrinsically low, possibly lower due to natural aging [24]. The FORS inflection point, therefore, either carries no information, or the information present is overwhelmed by the combination of instrumental variance and swatch-to-swatch variance. Elemental information does not seem to correlate with the FORS inflection points range, but there is separation, so that separation might be related to concentration. A concentration-dependent inflection point can then be rationalized because as concentration increases, reflectance of a diffuse swatch will decrease, causing a bathochromic shift at the inflection points: these would be detected by variance-based algorithms like PCA.

High performance liquid chromatography (HPLC) verification

HPLC analysis showed that all samples in the dataset (and principal component space) contained carminic acid, dcII, and dcIV, among other characteristic chromophores present in cochineal (Dactylopius coccus), much like previous analyses on red Norwich textiles [18]. Five dye combinations were identified: (1) cochineal alone (C), (2) cochineal and madder (CM) with alizarin and purpurin identified on most samples; (3) cochineal and young fustic (CF), associated with fisetin, fustin, and other flavonoids in Cotinus coggygria; (4) cochineal and brazilwood (CB), thanks to the presence of sappanol and urolithin C in Caesalpinia braziliensis; and (5) cochineal, madder, and young fustic (CMF) [2]. Spectra from each of the HPLC-identified “recipes” were analyzed for similarity of spectral features. CMF, CF, and CB were all uniform across spectra and samples in those identified recipes, while some inhomogeneity existed in CM and C. CM contained one sample that was spectrally more similar to CMF than CM and upon further investigation, it was found that that swatch was positioned next to a yellow textile that could contain young fustic, possibly explaining this outlier. Samples identified to be C showed the greatest spectral variance by far, but since all analyzed samples contained cochineal, this added variance should be expected.

When these samples are projected into principal component space, a similar trend is seen (Fig. 10)—samples containing cochineal project across much of the space, primarily in the green and blue clusters. The large blue cluster contains CM (except the outlier) and CB; CF lies in its own cluster between the green and blue clusters, and the red cluster exclusively contains CMF. In the 2D PC space, HPLC-identified recipes seem to zone across different regions of the blue and green clusters.

Fig. 10

a PCA scores plot showing PCs 1 and 2 of the VIS region identical to Fig. 7b with HPLC-identified samples overlaid and colored based on identification—black (CM), cyan (CMF), orange (C), steel blue (CF), and purple (CB), and b a 3D scores plot of the same information showing PC scores 1–3

Examining the same data across PCs 1–3 (Fig. 10b), shows that no HPLC-identified clusters overlap in the PC space, but the linear separability of CM, CB, and C stands. This could indicate that the large blue cluster is a combination of multiple dye recipes that is unresolved in a space containing all 198 swatches; or it could be a function of the limited HPLC sample set that was analyzed. If the data space is reduced and PCA is applied to just swatches identified as “blue,” there is minimal additional zoning. Additional exploration of this is necessary, particularly focusing on data processing methods to better highlight spectral differences between swatches in this potential “super” class. Based on the HPLC identification, it can be concluded that FORS shows some promise for identifying dye recipes, but this conclusion should be taken with caution, as only a small subset of swatches were verified with HPLC, leaving much of the full structure of the underlying data unmapped.

Study limitations and future work

This study aimed to explore FORS as an alternative to HPLC, intending to enable more cultural institutions to assess their textile collections. FORS has many advantages, including its portability, relative smaller cost, and ease of interpretation; but its use in dyed textiles is limited at-present, in part because chromophores are near the detection limit of this instrument. While this work is promising, a number of limitations exist.

The first limitation is the narrow scope of the dataset (late 18th- early 19th c. red textiles from Norwich, UK), chosen due to their high quality and minimal observable aging. To address the potential effect of natural aging on this dataset (textiles were contained in swatch books), textiles were selected from four different swatch books. The second limitation is the proportion of samples validated with HPLC. Ideally, the full dataset of 204 samples would have been validated with HPLC, which would have expanded analysis capabilities to do classification instead of cluster analysis. However, sampling restrictions prevented a full validation and instead, swatches were selectively sampled to span the entire principal component space post-chemometric analysis to attempt to mitigate the small validation size.

Ongoing research is addressing textile aging, using freshly dyed textiles as a comparison point for this dataset. And to test whether FORS can distinguish minor spectral features or classifies exclusively on color information, colorimetric analysis coupled with PCA is underway, to assess classification success utilizing exclusively color data.

Conclusion

The present study intended to explore chemometric analysis of reflectance (FORS) and elemental (XRF) information to non-destructively analyze dyed textiles. Well-provenanced Norwich textiles were used to collect FORS and XRF datasets. To confirm what visualization of Principal Component Analysis (PCA) scores showed, selected textiles underwent micro-destructive HPLC analysis to confirm dyes/dye combinations. From the data presented here, the following conclusions can be drawn:

With respect to this dataset, FORS may provide discriminatory information about dyed textiles based primarily on the 380-469 nm region, which corresponds to the average human’s color vision—electronic transitions are detectable, but vibrational modes of chromophores are not. This is further suggested by the fact that HPLC-validated samples show distinct clustering in unique regions of the principal component space. At the current stage, it is unclear if dye recipe information is contained in minor differences of multiple spectral features (e.g., small spectral shifts), or if FORS is only able to detect color information. In the best case, spectral processing strategies must be revisited and optimized to best classify subtle spectral differences that correlate with dye recipe. In the worst case, having accurate spectral information would be a useful tool for confirmatory analysis and perhaps most relevantly, would expand access to other professionals with visual impairments, e.g., color blindness.

Elemental analysis via XRF indicated that the presence of elements associated to mordants es not correlate with the observed inflection point’s wavelengths of FORS spectra. This is either because the concentration of chromophores in textiles is low enough that the signal is overwhelmed by instrumental and swatch-to-swatch variance, or because FORS is unable to detect minor electronic and vibrational shifts based on structural changes associated with the presence or absence of a mordant in a textile. These data, coupled with HPLC analysis, suggests that inflection point shifts in FORS spectra were random and not correlated to concentration or type of chromophore in the analyzed textiles.

• This work yielded practical insights into the application of FORS for the analysis of textiles, but it should be noted that degradation of the textiles was not accounted for. Thus, further analysis is necessary to answer what effect would be observed on the models presented here from freshly dyed textiles relative to degraded textiles.

• The present work evidences the need for tailored data processing as necessary for multivariate analysis of the subtle spectral features that FORS provides. This is a knowledge gap that contrasts with the extensive documentation of electronic and vibrational transitions, coordination chemistry, and molecular interactions that relate FORS to spectral features.