Tiljander
Posted by Ari Jokimäki on June 28, 2010
Note (September 8, 2016): The article below contains some false statements. I have overlined the false parts. See the discussion below for further information.
Arthur Smith wrote about the Mann/Tiljander issue. For months I have had a draft about this issue waiting for me to finish and publish it. As Arthur Smith asks comments on this issue and as I do seem to have something relevant to say on this issue, it is time for me to finish this thing.
Mann et al. (2008, from hereafter I will call this paper “MEA”, proper references with links are given at the end of this article) used Lake Korttajärvi dataseries from Tiljander et al. (2003, from hereafter “TEA”) as temperature proxies in their reconstruction. There has been lot of different claims about Tiljander series being upside-down in MEA. McIntyre & McKitrick (2009, from hereafter “M&M”) wrote a comment to PNAS on MEA and on the Tiljander issue they said:
Their non-dendro network uses some data with the axes upside down, e.g., Korttajarvi sediments, which are also compromised by agricultural impact (M. Tiljander, personal communication),…
To this, Mann et al. (2009) responded:
The claim that “upside down” data were used is bizarre. Multivariate regression methods are insensitive to the sign of predictors. Screening, when used, employed one-sided tests only when a definite sign could be a priori reasoned on physical grounds. Potential nonclimatic influences on the Tiljander and other proxies were discussed in the SI, which showed that none of our central conclusions relied on their use.
McIntyre has also presented some claims relating to this before in his website. Here’s to my knowledge the first of his posts on the issue, saying:
By flipping the data opposite to the interpretation of Tiljander et al, Mann shows the Little Ice Age in Finland as being warmer than the MWP, 100% opposite to the interpretation of the authors and the paleoclimate evidence. The flipping is done because the increase in varve thickness due to construction and agricultural activities is interpreted by Mann et al as a “nonlocal statistical relationship” or “teleconnection” to world climate.
Is the TEA data upside-down in MEA?
One of the four dataseries in TEA is the relative X-ray density. This dataseries is presented in TEA Figure 5. MEA presents it (among some other proxies) in their Supporting Information Figure S9. Mann website has the data and the codes for their reconstruction available. The relative X-ray density dataseries is there in the “data” section as tiljander_2003_xraydenseave.ppd. I have made a graph out of this data presented in Figure 1. It is easy to see by comparing the values in X-axis and Y-axis to those in TEA Figure 5 that the data is in the ppd-file exactly same way as it is presented in TEA.
Figure 1. The relative X-ray density series in TEA.
Now that it is established that the data in MEA input (in the ppd-file) is in same way as in TEA, I shall now look at the reconstruction codes a little. The ppd-file is read among other proxy files and written to a data-matrix. TEA data is not handled in any special way compared to other proxies before it is entered to the matrix, so TEA data is not turned upside-down at this stage.
Data is then read from the data-matrix to perform the reconstruction (there are several steps involved in this but it is not important here). There’s nothing special done on TEA data at this point either compared to other proxies, so the data seems to stay the same way throughout the reconstruction. So, the Tiljander data clearly is not flipped upside-down there. That doesn’t mean the Tiljander data is handled correctly there. Tiljander data is actually handled upside-down there. It is because the data is given in TEA so that higher values of relative X-ray density correspond to lower temperature values, so MEA should have turned the data upside-down before using it in their analysis.
Seeing the real situation with this issue, it seems that MEA did an honest mistake, which of course should be corrected. There has been some claims that MEA would have done this deliberately, and/or that they have refused to correct the mistake. I don’t see any proof here that they would have done it deliberately. The thing about correcting the mistake has more to do with M&M description of the situation than MEA attitude, in my opinion. The way M&M descibed this problem is bizarre, just like Mann et al. (2009) put it. M&M comment to me seems so misleading that I don’t see how it could be considered as informing the situation to MEA.
Are McIntyre’s claims about the issue correct?
Looking at McIntyre’s claims on this and the real situation descibed above shows that McIntyre’s claims are false. Just look at the graphs McIntyre presented. In all graphs there the values in X- and Y-axes give matching values, and yet McIntyre shows them as they would show the mistake. The graphs he presents just show how the data is in the TEA and in the input of the MEA reconstruction so basically those graphs just show that MEA have not flipped the data upside-down before feeding it to their analysis, which is exactly the opposite that McIntyre claims to be the case. He claims that MEA have flipped the data, while in real world the problem is that they haven’t flipped the data.
McIntyre’s role in this is very strange. Even when the basic claim is correct, i.e. the claim that the data is used upside-down in MEA, McIntyre still cannot make correct claims about it. It makes one wonder. If a person is aware of this problem, then why make false claims about it?
Bizarre, indeed.
References
Mann, Michael E., Zhihua Zhang, Malcolm K. Hughes, Raymond S. Bradley, Sonya K. Miller, Scott Rutherford, and Fenbiao Ni, PNAS September 9, 2008 vol. 105 no. 36 13252-13257, doi: 10.1073/pnas.0805721105. [abstract, full text, supporting information]
Mann, Michael E., Raymond S. Bradley and Malcolm K. Hughes, Reply to McIntyre and McKitrick: Proxy-based temperature reconstructions are robust, PNAS February 10, 2009 vol. 106 no. 6 E11, doi: 10.1073/pnas.0812936106. [Full text]
McIntyre, Stephen, and Ross McKitrick, PNAS February 10, 2009 vol. 106 no. 6 E10, doi: 10.1073/pnas.0812509106. [Full text]
Tiljander, Mia, Matti Saarnisto, Antti Ojala, Timo Saarinen, A 3000-year palaeoenvironmental record from annually laminated sediment of Lake Korttajärvi, central Finland, Boreas, Volume 32, Number 4, December 2003 , pp. 566-577(12). [abstract, full text]
Appendix – Tiljander relative X-ray density against local temperature
UPDATE (June 29, 2010): I decided to add this section as there’s a question relating to this by AMac in the comment section.
I have also looked at the relative X-ray density series of TEA against the local temperature record. The closest temperature measurement site in GISS database is Jyväskylä (which is quite close to the Lake Korttajärvi – about 10 km). Here’s the link to the GISS-data of the Jyväskylä temperature measurements. Jyväskylä temperature series start only from 1950, so there’s not much overlap between the two series. I inverted the Tiljander series and plotted the two in the same graph. Here’s the result:
The linear correlation between the two series is only about -0.12, so the relative X-ray density is not very good temperature proxy. It is, however, interesting to see that this series (when inverted) tracks temperature somewhat even if there were strong human influence to 20th century part of the series (according to TEA).
AGW Observer 
AMac said
Thanks for considering the issue of the use of the Lake Korttajarvi varve data series in the Mann08 reconstructions.
It is of particular note that you state,
(I agree that there is no reason to see this error as other than an honest mistake.)
Sometimes it is useful to identify the key questions that underlie a contentious issue. I hope the defenders of Mann08’s analysis will plainly address these two plain queries. as related in Comment #132 of the recent discussion at Collide-a-scape, The Main Hindrance to Dialogue (and Detente).
And
I have walked through the use of the XRD proxy by Mann08 in the post, The Newly-Discovered Jarvykortta Proxy — II. Elsewhere on that website, I’ve compiled links to literature and blog posts that are relevant to this issue. You may find those useful.
Ari Jokimäki said
I’ll address the queries, but I don’t consider myself as a “defender of Mann08” – I have just argued against what I see as false claims on the issue.
I added an appendix section above where I show the relative X-ray density against local temperature. I don’t think it’s very good proxy but I guess temperature can be seen there. However, outside of the presented 1950-2000 period, the X-ray density series clearly has places where it doesn’t follow temperature, so I wouldn’t use this as temperature proxy.
I’m not quite sure what you mean. They did discuss the Tiljander proxies in the supporting information:
“These records include the four Tijander et al. (12) series used (see Fig. S9) for which the original authors note that human effects over the past few centuries unrelated to climate might impact records (the original paper states ‘‘Natural variability in the sediment record was disrupted by increased human impact in the catchment area at A.D. 1720.’’ and later, ‘‘In the case of Lake Korttajarvi it is a demanding task to calibrate the physical varve data we have collected against meteorological data, because human impacts have distorted the natural signal to varying extents’’). These issues are particularly significant because there are few proxy records, particularly in the temperature-screened dataset (see Fig. S9), available back through the 9th century. The Tijander et al. series constitute 4 of the 15 available Northern Hemisphere records before that point.
In addition there are three other records in our database with potential data quality problems, as noted in the database notes: Benson et al. (13) (Mono Lake): ‘‘Data after 1940 no good — water exported to CA;’’ Isdale (14) (fluorescence): ‘‘anthropogenic influence after 1870;’’ and McCulloch (15) (Ba/Ca): ‘‘anthropogenic influence after 1870’’.
We therefore performed additional analyses as in Fig. S7, but instead compaired the reconstructions both with and without the above seven potentially problematic series, as shown in Fig. S8.”
They are not silent about it and even tested the effect of those proxies. They had determined the selection criteria, which these proxies fulfilled, so in that sense I don’t see this as “highly unconventional”. One could argue here that perhaps their selection criteria is too loose. But in the end, sometimes mistakes like this just happen. I don’t think there’s not much relevant in this issue in the sense of scientific practices generally.
AMac said
Wow, fantastic work, Ari!
I’ve no time to comment now, but want to leave this link. It will lead to the CRUtemv3 temperature anomaly record for the gridbox containing Lake Korttajarvi. That is what was used, according to Mann08’s Methods.
Cheers, AMac
AMac said
Ari,
Your position has changed substantially since you and I communicated on the subject in November 2009, at the tail end of the Stoat thread “Tiljander,” Comments 101-113. This is very heartening–that a person can look at a narrow technical issue and judge it on its merits. So rare for AGW-related areas, for the membership of all the “tribes.”
Was there anything in particular that caused you to re-evaluate this question of the orientation of the Tiljander XRD proxy in Mann08’s analyses?
In Mia Tiljander’s 2003 paper, Figure 2 graphs the seasonal temperature records and the precipitation record for the Jyväskylä station, from 1881 through 1993. As you note, the NASA GISS page only contains information from 1950 forward. I have looked online for pre-1950 Jyväskylä temperature records, without success. Do you know if they are available?
In any case, Mann08’s methods state that for screening, validation, and calibration, they used the local 5 degree by 5 degree gridcell temperature anomaly record. The borehole is at 62.3318 N, 25.6821 E. So the relevant gridcell would be 60N-65N, 25E-30E. I pulled the monthly anomaly numbers for that cell from CRUtemv3; those are the figures in the “CRUtem monthly” worksheet in the file Korttajarvi_gridbox_CRUtemv3.xls that is downloadable from BitBucket.org.
(I see that I mislabeled the graph “55N-60N”; the data is correctly labeled “60N to 65N.” I will correct that typographical error.)
AMac said
I had some trouble uploading to BitBucket. The Excel file with the temperature anomaly data for the Lake Korttajarvi gridcell is now called Kort-gridbox-CRUtemv3.xls.
Arthur Smith said
Ari, thanks for this outline of what you see as the situation.
However, I don’t think the issue is how the data was input to Mann’s analysis at all. Upside down or right side up, multiplied by 100, shifted by -500, whatever, the point of Mann’s response to M&M seems to indicate these shifts would not matter to the reconstruction:
“Multivariate regression methods are insensitive to the sign of predictors.”
If they are using multivariate regression, that means they are creating a linear combination of the different series, and those linear constants multiplying each series can have whatever value or sign the regression picks out. It doesn’t matter if the input data is opposite in sign, the regression will just switch the sign of the fitted coefficient.
What I believe AMac’s real complaint is, and M&M’s garbled comment apparently indicated also, is that the correlation between temperature and this proxy is *reversed* in the modern (calibration) period due to contamination by non-climatic factors.
Now, I’m not sure what the evidence is for that. But the problem is not that Mann used it “upside down” or didn’t turn it upside down, the problem is (if this idea is correct) that the data series itself switches from being negatively correlated with temperature to being positively correlated.
Right? AMac, does that agree with your understanding of the issue? Ari, have you looked at this part of it at all?
MikeN said
>. In order to show warm values at the top of a graph, you need to invert the plot
McIntyre says the plot needs to be flipped.
This is separate from the contamination issue raised in the SI, as the contamination leads to using the proxy upside-down.
More significantly, you are able to recognize the error which Mann has not acknowledged, and repeated in 2009 in another Science paper.
AMac said
Arthur #7 –
> AMac, does that agree with your understanding of the issue?
“Upside-down” and “inverted” have a plain-English, colloquial meaning that fits Mann08’s use of the lightsum and XRD Tiljander proxies. For whatever set of reasons — good or bad — these terms have been seized upon, pulled, pushed, bended, and mutilated.
So let me describe the situation to the best of my understanding, without using those terms. Ari, MikeN, you can see if you agree.
In their paper, Tiljander et al described three characteristics of their lakebed sediment cores: darksum, lightsum, and XRD. Here’s a brief summary of what they claimed.
Darksum: values are higher during warmer years.
Lightsum: values are lower during warmer years.
XRD: values are lower during warmer years.
We can look at the influence of these three series on the reconstructions that were generated in Mann08. A higher value of darksum, or lightsum, or XRD does what for the year in question: make that year’s deduced paleotemperature anomaly seem higher, or lower? (If a record don’t have an influence on the deduced paleotemperature, then it isn’t serving as proxy to temperature — right?)
Review of the proxy records, and the relevant CRUtemv3 gridcell, and the Mann08 methods and SI, and Mann08 Fig. S9 — together, they show that these Tiljander proxies are used in the following fashion.
Darksum: values are higher during warmer years.
Lightsum: values are higher during warmer years.
XRD: values are higher during warmer years.
Thus, the following relationships, with respect to “what a higher proxy value signifies, qualitatively”:
Darksum: Mann08’s interpretation is in agreement with Tiljander03’s interpretation.
Lightsum: Mann08’s interpretation is opposite in direction from Tiljander03’s interpretation.
XRD: Mann08’s interpretation is opposite in direction from Tiljander03’s interpretation.
Why is this?
Because the Tiljander proxies are not calibratable to the instrumental record, 1850-1995. As stated in Tiljander03, the non-climate signals progressively swamp the climate signals in the proxies, starting around 1720 and continuing to the recent past (i.e. the 1990s). Those non-climate contaminating signals are:
Higher darksum values
Higher lightsum values
Higher XRD values
Mann08 inadvertently accomplished a faux calibration to the non-climate signal in each record.
As it happens, in the case of darksum, the faux calibration yields a faux relationship with temperature that is the same in direction with (though different in magnitude from) what a true calibration would have yielded, were it possible (accepting Tiljander03’s analysis).
As it happens, in the case of lightsum, the faux calibration yields a faux relationship with temperature that is opposite in direction with (as well as different in magnitude from) what a true calibration would have yielded, were it possible (accepting Tiljander03’s analysis).
The analysis of XRD is the same as for lightsum.
The fourth Tiljander proxy used by Mann08, thickness, is not interpreted in Tiljander03.
Arthur Smith said
AMac (#8) – right, well consider the “lightsum” case and let me restate what I think you’re saying so we can all be clear on it:
Pre-1720: “lightsum” values decrease as temperature increases, and vice versa, according to Tiljander’s analysis. (But what was the basis for this understanding?)
Post-1720: Both “lightsum” and temperature increase through the modern period, according to Tiljander the increase is due to non-climatic contamination of the signal.
Mann’s analysis determined the relation between “lightsum” and temperature by calibrating to the period 1880-1990 or so. Therefore, no matter whether the “lightsum” data entered the analysis “right side up” or “upside down”, the regression would have forced increasing “lightsum” values to correlate to increasing temperatures. And therefore the calibration would have forced the pre-1720 temperature result to look “upside down”.
I.e. the problem was (A) in the data itself, coupled with (B) Mann’s algorithm doing temperature calibration during a period where the correlation between data and temperature had reversed (and was “contaminated”) due to non-climatic reasons.
*OR* the problem may have been simply that Tiljander was wrong about the temperature correlation of these proxies. Does anybody understand exactly why the original paper thought it was in a particular direction, as AMac has stated?
AMac said
Arthur Smith #9 —
Your analysis is largely correct.
Mann08’s overall instrumental period was 1850-1995. For validation, an early and a late period within this was established. I don’t recall the details; it’s laid out pretty clearly in the methods.
Here are quotes from Tiljander03.
MikeN said
Arthur, your statement is at odds with ClimateAudit. It is not in the relevant posts, but when I asked about it, I was told that Mann 08 algorithm won’t flip a proxy to match calibration. It has to be oriented manually.
Ari Jokimäki said
I’m little short on time, but I’ll comment quickly on few issues.
Well, some of my position there was caused by my misinterpretation of MEA S10, like I indicated in my last comment there. However, besides that, most of my comments are in line with what I’m saying here – that McIntyre’s claims on this are false no matter how the MEA algorithms treat the TEA proxies. McIntyre has quite clearly expressed that he thinks he sees the proxies being upside-down already in MEA S9 which just shows the proxies as they are before they are fed to MEA reconstruction. That claim is just wrong and it is the claim I have mostly been addressing.
I think there’s quite a lot of that actually but it tends to get buried by all the nonsense that is being poured on the public discussions on the subject.
I think it just continued from there. There also was a friend who was eager to dig deeper on the subject so I just browsed the codes a bit. Unfortunately I don’t have software needed for running the codes so I just read them.
They might be available for purchase from Finnish meteorological institute.
Ari Jokimäki said
Arthur, thanks for this. I was aware that the screening process might give TEA series a pass if it would use abslolute values for the correlation, but it didn’t cross my mind that the reconstruction algorithms might be sign-neutral too. That’s why I didn’t look there particularly, I just looked at the codes to see if they flipped TEA series there somewhere and didn’t find it.
I need to dig deeper to the codes to check that one out. Very interesting.
Well, I can’t know what really lies behind that M&M comment, but I know that McIntyre has been concentrating on the MEA figure S9 as if it would show that the data was flipped.
Like I indicated above, no, but will do it in the near future. 🙂
Gavin's Pussycat said
MikeN, are you sure about that? I checked Kaufman and there it is indeed the case that you have to input the individual proxy time series manually with the right orientation, because the scaling is done using the variances, which are always positive. In Mann et al it’s the instrumental-period trends that are used for scaling (calibration), which may be positive or negative.
Sure you (or your correspondant) are not mixing these two up?
AMac said
Ari —
(1) I don’t see any evidence in support of the idea that, as a group, skeptics (or skeptics & lukewarmers) are smarter than advocates of the AGW Consensus. With that in mind, go back to those three Stoat threads. Of the scientifically-literate skeptics/lukewarmers who commented, all understood the point that McIntyre and others were making, with respect to the orientation of certain Tiljander proxies. None of the scientifically-literate AGW Consensus supporters seemed to get that point. E.g. here (my annotation of the first thread).
I don’t think that this divide is explained by the notion that McIntyre’s remarks were “garbled” or “bizarre.”
(2) Last night, I read pages 61-64 of The Hockey Stick Illusion. Montford recounts McIntyre’s “audit” of the proxies used in MBH98. Prof. Mann’s group appears to have made many data-handling errors: in transcription, interpolation, infilling, endpoint handling, correction, use of up-to-date records, metadata handling, and version control. Such shortcomings have not been acknowledged by AGW Consensus advocates, much less rebutted, to my knowledge.
With that background, it seems likely that McIntyre began his analysis of Mann08 with the expectation that he would uncover data-handling errors.
In contrast, in my opinion, Prof. Mann has cast himself as the defender of climate science, vilified by the enemies of scientific progress. The unseen hand of a vast fossil-fuel-funded conspiracy is behind every criticism of him. E.g. in this interview by AGW Consensus advocate/journalist Chris Mooney (podcast, no transcript). The broader AGW Consensus community appears to accept this characterization.
People seemed to approach Mann08 in line with their preconceptions, and then stuck with them.
(3) In and of itself, the mishandling of lightsum and XRD by Mann08’s authors isn’t that important. What’s more significant is the way that institutions handled the affair, at various points along the way. In particular: the editorial and peer-review processes at PNAS. What did the reviews say? Were they cursory instead of thorough; if so, why? Did the editors ease the manuscript along, or did they attempt to demand improvements? Is McIntyre’s account of the obstacles placed in the path of his 2009 Comment correct? Why did the editors accept Mann et al’s non-responsive and misleading Reply as the final word on the matter?
These issues have yet to be examined.
Gavin's Pussycat said
AMac, I’m sorry my friend, you seem like a reasonable bloke but as long as you accept Montford (Bishop Hill) as a trustworthy source on anything, we’ll continue to have unbridgeable differences… your characterization of McIntyre’s comment text as, well, non-bizarre and requiring an answer other than that given, is itself bizarre. I honestly hope you find your way back to the real world some day.
AMac said
Gavin’s Pussycat,
Well, some people are impeccably reliable and others are almost never right. Most fall somewhere in the middle. Being somewhat new to the AGW Wars, it’s my blessing or curse to have started looking at Tiljander with few assumptions on that score. So far in “The Hockey Stick Illusion” (to page 64), I haven’t encountered any obvious misstatements that make my Spidey-Senses tingle.
That’s not the same as taking all of Montford’s assertions as indisputably true. Per Arthur Smith’s recent identification of an error made by Steve Mosher in his book, we can assume that “Hockey Stick” does contain mistakes. Reviewers often point them out with glee–and that’s in favorable reviews.
If you (and Ari) are interested, I could type in the account I described in #15 supra — it’s short enough to be covered under Fair Use. Then we could look to the MBH98 SI and online data archives (with side trips to RealClimate and ClimateAudit, no doubt) to see how Montford’s specific claims of data-handling errors stack up. Let me know.
As far as “McIntyre’s text,” the point I wished to make is that I could quickly grasp the arguments being made about Tiljander. I recall first haring about them on 10/26/09. You can see the “proxies-mistakenly-calibrated” argument as I made it at Stoat circa 10/27/09 – 10/29/09. I’m no climate scientist and I’m no genius. So it seems reasonable to suggest that McIntyre’s grammar ought not to have been an unbridgeable barrier to others’ understanding, either.
James Lane said
The orientation of the series input into the multivariate procedure is irrelevant. This is what Mann means when he says “multivariate regression methods are insensitive to the sign of predictors”.
The algorithm will “flip” the series if this provides a better correlation with the instrumental record, which is what happened in the case of Tiljander.
It’s had to imagine that Mann et al were unaware that the orientation of some of the Tiljander proxies in the output was the opposite to the interpretation given by the originating authors. When conducting multivariate regression is is incumbent on the researcher to examine how the algorithm has “signed” each series, to check the “face validity” of the output.
In this case the “face validity” of some Tiljander proxies is suspect, as they are inverted relative to Mia Tiljander’s interpretation. This should be discussed in the text of the paper. If you want to argue that Tiljander’s interpretation is wrong, fine, do so. But Mann et al don’t do that. They talk about the problems with the proxy post 18th century (which is likely the cause of the inversion, but don’t mention the inversion, and then perform a sensitivity test which shows that the problematic Tiljander series “don’t matter”.
For me, the obvious question is why use the Tiljander proxies at all? If they “don’t matter”, and there are problems with them (problems bound to be exacerbated in mutivariate regression), why not drop them altogether?
The last part of this post speaks to motive, so those that don’t care to speculate might like to skip it.
It seems to me that Mann et al needed Tiljander to “save” the “no-dendro” sensitivity test. They perform a “no-Tiljander” sensitivity test, and a “no-dendro” sensitivity test, but not a test without both Tiljander and dendro.
McIntyre has performed such a test, and assuming his emulation of Mann 08 is accurate, it does not deliver a similar reconstruction.
Here is McIntyre’s emulation:
– J
Ari Jokimäki said
If that is true, then it seems that MEA have it correctly in their analysis:
– The series is inputted as negatively correlating with temperature like it is in TEA
– The series seems to correlate negatively also with instrumental record (at least I calculated that with GISS series it is indeed negatively correlating as I also showed above)
– The algorithm notices it correlates negatively and then flips it
– The series is in the analysis as it should be so MEA have handled it correctly
James Lane said
Ari,
Not at all. The algorithm “flips” the orientation because it correlates with the recent temperature record. But we know this is suspect because the proxies are contaminated by human impacts post 18th century as noted by Mann (2008).
Read my post again. Where is the face validity? If you want to argue that Tiljander, got it wrong, do so. Mann et al didn’t.
Ari Jokimäki said
Based on Tiljander et al. we know that the relative X-ray density values have negative correlation with temperature (higher values mean lower temperature). This means that in order to use it with other proxies in a reconstruction, the series needs to be flipped. If the algorithm flips it, then it is exactly as it should be and Mann et al. haven’t done anything wrong in that sense – the series is not the wrong way around in the analysis.
Like I said, I calculated the correlation with the recent temperature record (using GISS data from nearby station) and got correlation of about -0.12. That is negative correlation (although not very good one) and according to you the algorithm will then flip it. All this would mean that all claims about Tiljander series being upside-down in Mann et al. would be false. Very interesting. 🙂
AMac said
James Lane #18 —
From the comment thread of Arthur’s post, here is my different guess as to “motive”.
Interestingly, this is largely a paraphrase of an AGW Consensus advocate’s idea–although he subsequently seems to have retreated from this theory.
AMac said
Ari & James —
The discussion of the orientations of the lightsum and XRD proxies as interpreted by Tiljander and as employed by Mann08 often appears to run into semantic difficulties. This is unexpected, as words such as “flipped” “inverted,”and “upside-down” have clear (and relevant) meanings in everyday English. But there it is.
It is more cumbersome but perhaps more useful to often refer back to the specifics of the interpretation. In other words, Tiljander’s view is clearly (see Tiljander (Boreas, 2003) and text quoted in #10 above):
lightsum — pre-1720, higher values generally correspond to lower temperatures.
XRD — pre-1720, higher values generally correspond to lower temperatures.
In addition,
darksum — pre-1720, higher values generally correspond to higher temperatures.
thickness — no explicit interpretation is offered.
James’ introduction of the term “face validity” (#18) is very useful. In other words, the ultimate effect of the introduction of a given proxy to the final paleotemperature reconstruction must be one of the following:
1. For all years, a higher value of the proxy causes the reconstruction to report a higher temperature (than would be the case if that proxy had a lower value for that year).
2. For all years, a higher proxy value causes the reconstruction to report a lower temperature.
3. For some years, a higher proxy value causes the reconstruction to report a higher temperature. For other years, a lower proxy value causes a higher reported temperature.
4. The value of the proxy in question has no effect on the reported temperature.
#1 is consistent with Tiljander’s interpretation of darksum.
#2 is consistent with Tiljander’s interpretation of lightsum and XRD.
#3 is a non-intuitive approach that reconstruction authors should disclose and discuss in their Methods, if it is used. (To my knowledge, there is no evidence from either supporters or critics of Mann08 that the paper employed such an approach.)
#4 is a trivial possibility, whereby the proxy has no effect.
It seems clear that the ultimate effect of lightsum and XRD on the Mann08 reconstructions is that higher proxy values cause the reconstruction to report a higher temperature. Thus, the “face validity” question is raised.
Ari — Do you agree with this analysis? If not, with what part do you disagree?
AMac said
Ari #21 —
Mann08 doesn’t screen, validate, or calibrate to local temperature records, according to their Methods. Rather, the 5 degree by 5 degree “gridcell” for each proxy is located, and the temperature anomaly record 1850-1995 for that gridcell is pulled from the CRUtem model. This CRUtem record is calculated mainly from primary records of stations within that gridcell, so gridcell and local records should have a high correlation. However, gridcell values are also influenced by nearby gridcell values, and those may have a significant influence when local records are sparse or discontinuous.
Thus, the temperature anomaly record for the gridcell covering {60-65N/25-30E} for 1850-1995 should be used in preference to local records, when looking back to the methods used by Mann08’s authors. That record is graphed in this post. The CRUtemv3 values for that gridcell can be downloaded from the CRU website (with difficulty); they are in an Excel spreadsheet linked from that post.
MikeN said
Gavin’s Pussycat, The question and response were definitely with regards to Mann et al, and not Kaufmann. That paper may not have even been published at the time. I haven’t personally checked this, but will attempt to do so soon.
MikeN said
It certainly looks like Mann’s response to McIntyre’s comment is stating the opposite.
AMac said
A follow-on post by Arthur Smith is up at Not Spaghetti, Michael Mann’s errors.
Ari Jokimäki said
I disagree with it in that you describe how they affect the reconstruction. That is something at least I don’t currently know (anymore 🙂 ). I find Tiljander et al. descriptions of these proxies satisfactory, so I’m using the orientation they describe. Like I have said above, the relative X-ray density has higher values for lower temperatures and lower values for higher temperatures. The way it affects the reconstruction is something to be determined. If it really correlates negatively with recent temperatures and the algorithm really flips the negatively correlating proxies, then the orientation of that proxy is in the reconstruction as it should be (whether it’s a good temperature proxy or not is another question).
Those are the two things I need to find out: 1) the correlation with recent temperatures as used in MEA, 2) a verification that algorithm flips negatively correlating proxies. Well, ok, I also need to find time to check these two things so that’s three things I need to find: the flipping, the correlation, the time and the will to do it… err… so that’s four things I need to find: the flipping, the…
When I have found those things, then we’ll do the Finnish inquisition.
MikeN said
These threads went quiet. Looking at the matlab code released with the paper, I think the algorithm will not flip the proxy when negatively correlated.
if (z(3,i)==9000 | z(3,i)==8000 | z(3,i)==7500 | z(3,i)==4000 | z(3,i)==3000 | z(3,i)==2000) &…
z(1,i)>=ilon1 & z(1,i)=ilat1 & z(2,i)=corra
…
temp=x(kk:kkk,i+1)*sign(z(ia,i));
…
corra =.106, so a negatively correlated proxy would be dropped. The sign(z(ia,i)) will always be +1
There are other sections of code with abs(z(ia,i))>=correlation, but Tiljander is class 4000
It may be that they left out an abs here, and Mann just thinks the regression is blind to the sign.
AMac said
MikeN,
On July 1, Arthur remarked
I guess Ari’s thinking things over, too.
As far as the matlab code, can you interpret the meaning of “the algorithm will not flip the proxy when negatively correlated” (assuming it’s true)?
IT seems to me that the Mann08 procedure will establish a faux calibration for each of the four Tiljander proxies. In each case, the orientation of the proxy value to gridcell temperature will be {higher proxy value} corresponds to {higher temperature}.
With respect to Tiljander03, this oriention would be correct for darksum, wrong for lightsum and XRD, and indeterminate for thickness. (The slope of the faux calibration would likely be incorrect in all four cases.)
So, in Mann08, for the authors and for the code, what condition would define a “negatively correlated” Tiljander proxy? Which of the proxies might be subjected to “flipping”?
Ari Jokimäki said
Actually I haven’t pursued this very actively lately. I did peek the codes again and wondered about the sign-function there. It’s possible that MikeN’s analysis is correct. However, in that case the relative X-ray density (for example) would not be included to the reconstruction although I still need to check that the correlation is indeed negative.
The mentions of the abs function in the code suggest that they might have been using absolute correlation values but dropped them at some point.
Tuomas Helin said
Arthur, AMac, MikeN (& Ari):
For your convinience, I suggest you to contact one of the few people who tested the algorithm before McIntyre wrote the comment to PNAS:
jean_sbls[ät]yahoo.com
JeanS is a co-author in ClimateAudit-blog and has been willing to respond to e-mailed questions related to the algorithm. Will save a lot of your time to discuss the open questions with a person already familiar with those issues, I think. Ari already has some of JeanS’s e-mails on the topic in Finnish.
I’m proposing this as I’m highly interested to hear if you would reach the same conclusions as these statisticians that are a bit on the skeptical side. I’m quite sure that the algorithm really treated the Tiljander data upside down by coincidence, because so many persons (Kaufman and two Finnish statisticians) have agreed with McIntyre.
So regarding to the open questions presented in , answer to the question (1) seems to be yes. Regarding to the question (2), McIntyre doesn’t seem to understand that he hasn’t been clear in presenting that the Tiljander-data was interpreted upside down in Mann’s reconstructions. So before autumn 2009, it’s not a clear yes nor a clear no.
What would be most interesting to know is the answer to the question (3). Visual interpretation of figures suggests that no, but JeanS claimed the opposite in a personal communication.
Am happy to see that you are digging into these questions. As I said before, I think it will save you a lot of time to communicate with people familiar with the algorithms and then verify/disprove their conclusions.
Cheers,
-Tuomas
AMac said
Ari #31,
> However, in that case the relative X-ray density (for example) would not be included to the reconstruction
I am looking at the correlation between XRD and gridcell temperature for 1850-1995 (as plotted here per Comment #3). Mann08’s authors recognized from the Tiljander03 text that there could be modern contamination problems, then decided that XRD and the other three proxies could be used. (Per the Methods in the SI–I think this decision was a mistake, but that’s what they said they did.)
Given that choice, XRD (and lightsum) appear to be the same as the other proxies used in the reconstruction, with respect to the way that the Mann08 procedure would approach the issue of its orientation. In other words, higher proxy values correspond to higher temperature.
This is why I am unclear about how some part of the code might have prevented the inclusion of XRD (and lightsum) into the reconstruction.
MikeN said
Tuomas, I think it is Jean who told me the result to begin with.
MikeN said
The Matlab code as is is not correct. I tried to cut a bit for brevity.
The if statement contains z(ia,i)>=corra.
So if a proxy is negatively correlated, then it should not enter into this piece of code. Other portions of code refer to abs(z(ia,i)).
Amac, one argument is that aside from the contamination issue, that if a proxy is entered upside-down into the algorithm, it will flip it back to correct orientation. I don’t think this is the case.
Ari Jokimäki said
A quick thought to add one dimension to this issue: there’s also a possibility that the code released in Mann’s website might not (by mistake) be the actual code used in the reconstruction. This might explain the rather strange comments in the code relating to the abs-function.
AMac said
Interest in Tiljander seems to have flagged a bit, here and at “Not Spaghetti.”
Anyway, it would be on-topic for me to point out that these series are presented graphically at this blog post, and the data themselves are downloadable as an Excel spreadsheet.
When one looks at the Tiljander series, it seems to me that one is obliged to conclude the following: they cannot be meaningfully calibrated to the instrumental temperature record, 1850-1995. During the 19th and 20th centuries, any climate signal in these series are overwhelmed by local non-climate influences. As discussed in Tiljander03, these are road building, farming, peat cutting, bridge reconstruction, and lake eutrophication. The correlation between each of the series and the CRUTEM3v gridcell temperature record are instances of “nonsense” correlations.
I belatedly realized two other things.
1. Mann08 performed a log transformation before correlating them to the CRUTEM3v temperature record. From a mechanistic point of view, I don’t know if this is valid: it is apparently a common thing to do in paleolimnology, though. Also, I strongly suspect that the distribution of these series have a lot of kurtosis, and are closer to log-normal than normal. That may be a separate justification for the tranform.
2. While Mann08 considered four series (Darksum, Lightsum, Thickness, and XRD), there are actually only three. Darksum is Lightsum subtracted from Thickness: thus, Thickness contains no information that is not already contained in Darksum and Lightsum. Mann08 was thus in error to consider Thickness as an independent measure. (This explains why Tiljander03 did not discuss the meaning of Thickness.)
MikeN said
A log transformation of series that are subtracted from each other? How can all three series be valid?
AMac said
Darksum, Lightsum, and Thickness are descriptions of varve thickness in millimeters, though Darksum and Lightsum are expressed as tenths of microns (i.e. as a 10,000x larger number). Darksum was derived as follows by Tiljander and colleagues:
Darksum = Thickness – Lightsum
Mann08’s authors screened the following four series against the CRUTEM3v records, 1850-1995: ln(Darksum), ln(Lightsum), ln(Thickness), and ln(XRD). The first three were validated, ln(XRD) was not. More here.