HOBBY GRADE TEST KITS CAN OUTPERFORM ICP MEASUREMENTS…REALLY??

[Reefahholic]

@taricha

I also want to show you what I typically see when somebody transitions over from ICP-OES to ICP-MS. This will look very similar to my other pictures, but it’s important to note this was recently. I highlighted the date for you. The elements I want you to pay close attention to are also highlighted. Remember when I said that there’s several trace metals that OES doesn’t have the sensitivity to detect until they up near 2-3 ug/L and even then the results are questionable.

So check it out and notice the dates.

His previous ATI ICP-OES before this one.

 

[Reefahholic]

As you can see above ^^^ Those elements were in there, but ATI could not detect them at those lower levels. They are beyond the LLOD.

 

[Thales]

@taricha

I also want to show you what I typically see when somebody transitions over from ICP-OES to ICP-MS. This will look very similar to my other pictures, but it’s important to note this was recently. I highlighted the date for you. The elements I want you to pay close attention to are also highlighted. Remember when I said that there’s several trace metals that OES doesn’t have the sensitivity to detect until they up near 2-3 ug/L and even then the results are questionable.

So check it out and notice the dates.

His previous ATI ICP-OES before this one.

How are you determining that the MS results are accurate and precise?

 

[BeanAnimal]

That is the part that I can’t get past. Pages and pages of screen shots touting what can be gleaned by comparing different ICP tests and highlighting values, all based on the assumption that the favored one is correct and the others are not.

This thread attempted to clarify the accuracy by spiking samples with known quantities. The results appear to indicate that ICP results are not to Be assumed accurate.

 

[areefer01]

So we’re these both OES? It’s either that or AFR isn’t keeping up with your tank. Iodine looks good though.

I don't know the details behind Oceanmo or ATI other than ICP. I am sorry. As far as AFR keeping up interesting point and nothing I've actually looked at. I am mainly using it to maintain ALK, Ca, and Mg since it is a single dosing solution.

I think there have been a few points of discussion around AFR and elements in a couple different threads. With regards to iodine Tropic Marin had increased that a year or so back.

In any case didn't mean to cause more noise. I only added that chart as the it came up that Oceamo may show more elements than ATI so I was curious.

BTW - good morning everyone.

 

[Pod_01]

It’s a little odd that it’s staying stable w/o any supplementation.

I should clarify, I used TM AFR up to about 4 months ago so it was supplemented.
About 4 months ago I switched back to FM Balling Light so the trace elements are supplemented by the base system and Cu is keeping steady at 3 or so as per Fauna Marin recommendations.

I used the Balling Light system way back when I started my Red Sea Reefer 250 and it worked but I got lost by following the various forums recipes/ideas/suggestions like dose this, measure that, keep eye on this/use GFO etc… and that didn’t help etc… So for few years I went to AFR simple system and I tuned out the noise and just tried to learn how to reef, by selectively listening to 2 or 3 people at most.

Things are looking lot better these days:

It is far from perfect but I am getting there.

 

[Rick Mathew]

How are you determining that the MS results are accurate and precise?

My question exactly.

 

[MnFish1]

My question exactly.

IMHO - to answer that question one would need to know what methods the companies themselves used to create the ranges that they say are 'green'. Did they use seawater, spiked fresh water, etc etc. Did they pick 'normal ranges' that seemed to do better with coral than 'Seawater'? I.e. was the goal 'matching seawater' or was the goal 'the best amounts for growing coral (i.e. their opinion). Unless this is known, it's impossible (IMHO) - to determine whether the results from one company vs another is 'more correct' or not?

 

[BeanAnimal]

IMHO - to answer that question one would need to know what methods the companies themselves used to create the ranges that they say are 'green'. Did they use seawater, spiked fresh water, etc etc. Did they pick 'normal ranges' that seemed to do better with coral than 'Seawater'? I.e. was the goal 'matching seawater' or was the goal 'the best amounts for growing coral (i.e. their opinion). Unless this is known, it's impossible (IMHO) - to determine whether the results from one company vs another is 'more correct' or not?

I think you kind of lost the conversation (understandable if you jumped in late and didn't read all 500 pages). The bulk of this conversation and Ricks quoted comment are not about the vendor recommended ranges or suggestions. We are discussing use of the measured values returned from the test vs the actual concentrations in the sample and how those values are being trusted for whatever method or conclusion is being drawn.

Reefaholic (representing a group of moonshiners proponents) trusts the ICP-MS values as a guide and has illustrated that he feels that are "valid" by comparing them to other ICP methods. But the results from the test presented in the first posts of this thread indicate that maybe none of them should be trusted at this level.

The subject of vendor recommend safe ranges is somewhat outside of the conversation, as it is a whole different subject.

 

[jda]

Since nobody seemed to have any issue with my list of facts or truths or whatever, should we apply the same amount of skepticism to ICP companies doing titration or color changing tests that some have applied to the LFS?

It is reasonably common advice around here for folks to get their own test kits and not trust the LFS employees doing the work. Why not the same for some unknown individual at an ICP company?

If you want to just trust a rando counting drops, then why not just use the LFS and save your money on ICP for testing compounds?

Again, this was the some of the point of this whole thread, right?

 

[MnFish1]

I think you kind of lost the conversation. The bulk of this conversation and Ricks quoted comment are not about the vendor recommended ranges or suggestions. We are discussing use of the measured values returned from the test vs the actual concentrations in the sample and how those values are being trusted for whatever method or conclusion is being drawn.

Reefaholic (representing a group of moonshiners proponents) trusts the ICP-MS values as a guide and has illustrated that he feels that are "valid" by comparing them to other ICP methods. But the results from the test presented in the first posts of this thread indicate that maybe none of them should be trusted at this level.

With all due respect I don't think I lost anything. There have been multiple topics discussed - including the reasons that 1 ICP provider's numbers may be vary considerably from another's. I believe one of the conclusions of the experiment was that trend following is more valuable than the actual numbers. I believe another conclusion was that using the same provider may be more helpful in doing so. One issue that is being ignored is that it's not the actual 'value' that these companies are selling, its their interpretation of the results they obtain. I would hope that there would be at least some rationale why one company recommends a copper level of x and another company recommends a copper level which is much different.

 

[Rick Mathew]

Since nobody seemed to have any issue with my list of facts or truths or whatever, should we apply the same amount of skepticism to ICP companies doing titration or color changing tests that some have applied to the LFS?

It is reasonably common advice around here for folks to get their own test kits and not trust the LFS employees doing the work. Why not the same for some unknown individual at an ICP company?

If you want to just trust a rando counting drops, then why not just use the LFS and save your money on ICP for testing compounds?

Again, this was the some of the point of this whole thread, right?

Actually interesting question you asked about LFS... Thank you for asking...MY answer would be the same for them as it would for ICP-OES-MS or whatever method one uses to measure... What is their measurement variability for any given measurement. I have several LFS in my area and I put them to the test for Ca , Mg, Alk, PO4 and NO3...in a very similar "Round Robin" experiment that is posted here and lets just say with one notable exception their performance supported your "commonly" held belief....Mostly a random number generator ....Measurement variability is everywhere...LFS, my testing, your testing, ICP-XXX, even in the gas pumps we use to fill our vehicles....The question is how large is it and how close is it to the true value...The answer to these questions lets me know how to use the reported value.

My 6 Sigma instructor said the most important question is not what is the variability but what are you going to use the measurement for? Then the variability has meaning, otherwise it is just numbers....By the way this has a name "Fitness For Use"

 

[jda]

The general stigma about LFS testing is that you don't know the skill, technique of the tester or often the type of test used. Basically a lack of transparency and unknown about the tester and the kit. This is no different than the ICP company, right?

I have been testing for 30 years. I have no idea if I should trust myself or not, but I do know whom I can blame if I mess up. I can also test again if I get a suspect result. I also have made my own conclusions about which test kits are the most accurate over time and stuff. I have accountability in my own home even though I am a moron.

Yes, there are variables everywhere. However, should we eliminate one variable and treat out of house titration and color changing tests as all the same? Why differentiate between LFS and ICP company results with these tests?

Perhaps all advice on ICP should be split between the desire for elemental vs compound analysis? I think that if you polled the majority of users, they would think that the ICP company uses the plasma to measure po4, for example.

This is purely anecdotal, but the last test that I saw at the LFS was a timed color changing test. They mixed it up and then spent 15-20 minutes packing new fish and frags. Then they looked at the test. Maybe it did not matter, but IME, it does. Ummm...

I had to take chaos theory, early 6 sigma stuff, learned how to actually figure a beta coefficient (any risk, not just investing), etc. in college and the most important thing that I remember from it is that doing it yourself rather than outsourcing to somebody else is perhaps more important than what you find from the analysis. I forgot most of the rest unless I open one of my textbooks again. Beyond fitness for use, I would consider "fitness of the fitter" to be more important... or in other industries, "evaluation of the evaluator" is paramount.

 

[jda]

Kinda in the same vein as above, are we sure that MS vs OES is really the difference in anything, or is Christoph just better than others? Given that there is no actual evidence that one type is better than another even if more granular, why not him?

 

[Pod_01]

If you want to just trust a rando counting drops, then why not just use the LFS and save your money on ICP for testing compounds?

Are you sure they have a lab person counting drops in Europe (Germany / Austria). I hate to say it people there do not work for $5 an hour. That would be time and cost prohibitive. If they do, WOW what a deal….

Do you actually know how they perform non ICP tests in these particular labs?

I just don’t see them doing it manually.

 

[Thales]

The testing issue has been around forever. Either do it yourself the same way every time, or find someplace that you trust for whatever reason and just stick with that Comparing different testing vendors will drive you crazy and mostly add to confusion rather than adding clarity. In my 2014 phosphate Macna talk I talked about this and how the some of results in the talk were from the lab at CAS, but more importantly, by the same person in the lab who I knew was a stickler for the protocol they liked. Since we really don't have any information on who or how any of the testing companies are doing much of anything, our choice comes inside a large black box inside several other black boxes.

I consider testing in this hobby a matter of trending, not accuracy, so I care about precision more than accuracy, and like things like the trident because it takes some variability out of the equation.

 

[Reefahholic]

How are you determining that the MS results are accurate and precise?

How do you determine accuracy with your 2-part dosing, CaRx, or hobby grade test kits?

Christoph has tested his machines and does QC. When I dose .05-1 ug/L and that element increases as expected that’s how.

 

[Reefahholic]

My question exactly.

How are you determining perfect accuracy on anything in this hobby. Please elaborate!

 

[jda]

Have you not been reading? You really cannot. Which is why it is smart to think about why you should pay a service for this... or more importantly fanboy for any service. As you know, I stopped recommending ICP years ago when they always seemed to return more questions than answer, so I don't know that recommending ICP is all that smart either.

Better to just help people understand what each type of testing is, accurately represent what they are and see if the hobbyist has any need.

As an aside, nobody has to give any answers as to what they do in their home or compare them to people who are selling services. I have never made any claim that what I do to make sure that my tank alk is accurate is the best (alk is about all that I test) to get $60 out of somebody as a superior method.

 

[Reefahholic]

How are you determining that the MS results are accurate and precise?

I consider testing in this hobby a matter of trending, not accuracy, so I care about precision more than accuracy

Lol.