Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Portables (Apple) Apple Hardware IT Technology

Consumer Reports Stands By Its Verdict, Won't Recommend Apple's MacBook Pro (mashable.com) 268

Consumer Reports took many by surprise last week -- certainly Apple -- when it said it doesn't recommend the company's new MacBook Pro models. The American magazine, which has garnered credibility over 80 years of its existence, said battery life on Apple's new laptops was all over the place -- hitting 19 hours in a test, but less than four hours in another. Last week, Apple's VP of Marketing, Phil Schiller insisted that Consumer Reports' findings didn't match the company's field data, and that Apple was working with Consumer Reports to understand its review. Now Consumer Reports has responded: The nonprofit organization is standing by its initial verdict in which it did not give the MacBook Pro (2016) its "recommended" rating. The organization has now said it doesn't think re-running the tests will change anything. "In this case, we don't believe re-running the tests are warranted for several reasons. First, as we point out in our original article, experiencing very high battery life on MacBooks is not unusual for us -- in fact we had a model in our comparative tests that got 19 hours," it said. "Second, we confirmed our brightness with three different meters, so we feel confident in our findings using this equipment. Finally, we monitor our tests very closely. There is an entry logged every minute, so we know from these entries that the app worked correctly," it added.
This discussion has been archived. No new comments can be posted.

Consumer Reports Stands By Its Verdict, Won't Recommend Apple's MacBook Pro

Comments Filter:
  • by Anonymous Coward on Friday December 30, 2016 @01:52PM (#53580193)

    Consumer Reports has no incentive but to produce accurate reports on consumer products. Apple on the other hand has a motive to produce positive results with its product tests. But this is not the first time Apple has over inflated battery life and I am sure it's tests were done to provide a good specification under certain conditions. But my own experience with devices today has tended to be overly optimistic battery life tested under not so realistic conditions. Consumer Reports has always provided more accurate battery life results.

    • by 93 Escort Wagon ( 326346 ) on Friday December 30, 2016 @02:08PM (#53580327)

      The problem is not "over inflated battery life" - and actually, Apple has (in the past) gotten kudos for being one of the few companies that consistently provided reasonably accurate battery numbers for their products.

      No, the issue is there's something as-yet-unexplained which, under some circumstances, causes the battery life of the newest MacBook Pros to plummet to ridiculously low levels. Consumer Reports saw it in their testing; but, even before that, some customers were experiencing it (and justifiably complaining).

      • Re: (Score:3, Informative)

        The problem is not "over inflated battery life" - and actually, Apple has (in the past) gotten kudos for being one of the few companies that consistently provided reasonably accurate battery numbers for their products.

        No, the issue is there's something as-yet-unexplained which, under some circumstances, causes the battery life of the newest MacBook Pros to plummet to ridiculously low levels. Consumer Reports saw it in their testing; but, even before that, some customers were experiencing it (and justifiably complaining).

        See my Post here [slashdot.org], for a possible cause.

    • Re: (Score:2, Interesting)

      by brausch ( 51013 )

      from Consumer Reports: "said battery life on the new MacBook Pro was all over the place, hitting 19 hours in a test, but less than four hours in another. "

      Seems like if they can't get consistent answers they would want to find out why?

      • by Anonymous Coward

        They're not Apple's QA department. Why should they care why the new Macs aren't performing acceptability? All they need to know is that they don't. It's Apple's job to find out why and fix it.

        This is of course one of the many dangers of using an OS with only one hardware supplier. I can't understand how anyone could not see what a stupid idea that is...

      • by Anonymous Coward on Friday December 30, 2016 @02:23PM (#53580443)

        It's not CR's job to find out why the product isn't working as expected/advertised. Their only job is to test the system in controlled and repeatable ways that can be demonstrated and are consistent with current quality assurance methods, and then to report on those tests to their paying subscribers. CR does not take money from anyone but their subscribers and buys off-the-shelf/lot products in order to ensure that there is no appearance of impropriety.

        In this case, they were comfortable enough with their results, even after Apple contacted them, to keep them. If they felt that the consistency was in issue with the tests (the same tests/test-systems that are run/used on other computer systems) then they would have stated that and reworked the tests. They have done this in the past when their tests were not working as expected.

        • It's not CR's job to find out why the product isn't working as expected/advertised. Their only job is to test the system in controlled and repeatable ways that can be demonstrated and are consistent with current quality assurance methods, and then to report on those tests to their paying subscribers. CR does not take money from anyone but their subscribers and buys off-the-shelf/lot products in order to ensure that there is no appearance of impropriety.

          In this case, they were comfortable enough with their results, even after Apple contacted them, to keep them. If they felt that the consistency was in issue with the tests (the same tests/test-systems that are run/used on other computer systems) then they would have stated that and reworked the tests. They have done this in the past when their tests were not working as expected.

          They're getting inconsistent results here, so either there's a flaw in the test, or there's a very intermittent issue. And given that (i) CR hasn't disclosed their full test protocol, and (ii) CR apparently did a second test using Chrome on the Mac and got consistently high results (ruling out hardware issues), there are very good reasons to suggest there's a flaw in the test.

      • by msauve ( 701917 )
        "Seems like if they can't get consistent answers they would want to find out why?"

        It's not their responsibility to figure out why the answers are different, as long as the questions are consistent. If the results are inconsistent, that's Apple's problem.
      • Why? They're just reporting how devices work to consumers. If i buy a car and it gives me 4hs on a full tank most people, me included, don't want to understand the mechanical reason why it happens.

      • by sjames ( 1099 )

        Not their problem. Their job is to report what consumers might expect from normal usage. Apparently that would be battery life all over the place.

        It's Apple's job to figure that out and improve the battery life. I'm sure CR will dutifully test an updated model and report on it.

    • Consumer Reports has no incentive but to produce accurate reports on consumer products.

      You mean other than this being their core business premise?

      The stupid is strong with this one.

  • by SuperKendall ( 25149 ) on Friday December 30, 2016 @02:09PM (#53580329)

    Consumer Reports, as they said, is pretty careful with testing. But even if they were not quite as careful as they are, as long as they tested different devices in the same way and used consumer purchased models, they results they found should stand.

    Hopefully Apple will get to the bottom of what happened in the tests, and make the laptops better. Then they can get back on the list next year. It does seem like some mix of software and hardware has some quirk if you can find the range of times Consumer Reports found.

    One thing I wonder is if it will even have much of an effect. Do many people really rely on consumer reports for laptop info? It seems like there are so many other sites comparing laptop hardware, that consumer reports is just one of many data points...

    And for Apple in particular that matters even less, because if you want a MacBook Pro you are buying what they are selling. It may mean someone would wait another year. Or it might mean that you would possibly purchased an older model instead (I had read somewhere that refurbished 2015 MacBook Pros were selling really well).

    I think Apple will iron this out within a month or so and then it really will not matter, but it makes me think more of Consumer Reports that they are willing to stick by results as they found them and not cave into pressure for a re-test.

    • by ilsaloving ( 1534307 ) on Friday December 30, 2016 @02:20PM (#53580421)

      I've been waiting since 2011 to upgrade but every model they put out has been more and more retarded. Soldered memory. Proprietary storage. Removing ports even when it destroys compatibility between the few products in their own meagre lineup.

      Apple has basically abandoned the professional market, and are now exclusively catering to rich people who sit all day in Starbucks looking at Facebook. I can think of no other reason for their direction in the last few years.

      • by Solandri ( 704621 ) on Friday December 30, 2016 @02:41PM (#53580573)

        I've been waiting since 2011 to upgrade but every model they put out has been more and more retarded. Soldered memory. Proprietary storage.

        You're behind the times. The newest Macbook Pros have their NAND storage soldered to the mainboard [ifixit.com].

        The previous iteration of their proprietary SSD had encrypted communications. It took OWC over a year to reverse-engineer it and offer compatible SSD upgrades. I guess Apple took that as a sign that they needed to eliminate any possibility of a third party upgrade. After all, you can't have customers modifying their hardware to their liking.

        • by ilsaloving ( 1534307 ) on Friday December 30, 2016 @02:58PM (#53580711)

          Oh FFS...

        • I've been waiting since 2011 to upgrade but every model they put out has been more and more retarded. Soldered memory. Proprietary storage.

          You're behind the times. The newest Macbook Pros have their NAND storage soldered to the mainboard [ifixit.com]. The previous iteration of their proprietary SSD had encrypted communications. It took OWC over a year to reverse-engineer it and offer compatible SSD upgrades. I guess Apple took that as a sign that they needed to eliminate any possibility of a third party upgrade. After all, you can't have customers modifying their hardware to their liking.

          Actually, it probably took OWC that long to get the Flash memory that was on Allocation due to commitments to Apple and others.

        • by AmiMoJo ( 196126 )

          Removed the SD card slot too, so no cheap and more or less flush secondary storage either.

        • The previous iteration of their proprietary SSD had encrypted communications. It took OWC over a year to reverse-engineer it and offer compatible SSD upgrades. I guess Apple took that as a sign that they needed to eliminate any possibility of a third party upgrade. After all, you can't have customers modifying their hardware to their liking.

          Not to mention forcing customers to pay for SSD storage at a rate of $800 / TB, more than twice what it would cost if purchased as a removable module from a vendor like

      • rich people who sit all day in Starbucks looking at Facebook.

        In which case they'll probably get a lot closer to the stated battery life.

    • Hopefully Apple will get to the bottom of what happened in the tests, and make the laptops better.

      Odds are good it is a software problem. Either that or a firmware problem that can be patched with software. I would look at the two GPUs and ensure they were not both turned on at the same time. But regardless of the cause, if they can get 19 hours after being patched then that makes for some amazing run times.

  • by TheFakeTimCook ( 4641057 ) on Friday December 30, 2016 @02:14PM (#53580365)
    Sounds like they have stopped being "objective" and have moved on to "defensive".

    HOWEVER, an interesting anecdote comes from reading another online forum (MacRumors.com), last evening, where a poster with a tbMBP 15" noted that, ONE TIME, when he unplugged an external Thunderbolt display (TB displays FORCE the MBP to use the dGPU), "Activity Monitor" said in the "Energy" tab that, instead of the 10 or 11 hours he was getting on average, it was showing that he was expected to get 3 hours.

    However, no Processes were showing as being Energy-Hogs, and, he also stated that the "CPU" Tab showed that nothing was using over 1.5% CPU (which was reasonable for what he had running). And what he did have running SHOULD (and probably was) running on the iGPU. (???)

    But, what was really "telling", was that he reported that the area under the "E" and "R" keys on the Keyboard was getting REALLY HOT. Hot enough that he panicked, and Rebooted the laptop.

    Everything returned to normal, battery life report back to normal, no heating, hasn't happened since...

    So, looking at the iFixit teardown of the 15" MBP [ifixit.com], you can see in Step 6, that the components that would be under that area of the Keyboard would plainly be the AMD GPU (outlined in Yellow) (and not the CPU, which is over nearer to the "I" and "O" keys, basically).

    So, something is (maybe) occasionally causing the AMD GPU, not the CPU, to run amok (or even be in some sort of power-guzzling "SCR-Lockup" state (hopefully not!)), sucking down the juice. Obviously, CR and others haven't triggered this behavior in the same way as the MacRumors poster; but there may be more software paths to this bug, likely involving switching between dGPU and iGPU modes, and/or power-savings involving same.

    More than likely this is still a software issue; but it is not one that Users can see in Activity Monitor (other than it does seem to "know" that the battery is being drained by something, hence the low "Time Remaining" number). Apparently, Activity Monitor doesn't report separately on GPU Energy usage (they need to change that!)

    Just an interesting little tidbit, that belies the assertion that a "retest" wouldn't make a difference (after Apple has a chance to address this issue, of course).
    • by Anonymous Coward on Friday December 30, 2016 @02:35PM (#53580521)
      Doesn't sound defensive to me. Why should they re-run the tests for one particular company?
      • by sinij ( 911942 )
        They should, if such company came along with "we fixed the issue you identified, please re-test updated product". However, this is not what Apple is doing.
    • by Luthair ( 847766 )
      Why should they continue to re-run tests until they get the results the vendor wants? Running and monitoring the tests isn't free and they have finite resources, it isn't their responsibility to do the vendors work for them and their tests have otherwise produced consistent results.
    • by dgatwood ( 11270 )

      So, something is (maybe) occasionally causing the AMD GPU, not the CPU, to run amok (or even be in some sort of power-guzzling "SCR-Lockup" state (hopefully not!)), sucking down the juice. Obviously, CR and others haven't triggered this behavior in the same way as the MacRumors poster; but there may be more software paths to this bug, likely involving switching between dGPU and iGPU modes, and/or power-savings involving same.

      I hope you're wrong. Otherwise, in two years, Apple is going to end up doing yet another logic board recall....

  • What that tells me (Score:5, Insightful)

    by Sydin ( 2598829 ) on Friday December 30, 2016 @02:16PM (#53580391)
    Is that Apple had no interest in actually sending the logs and test data to their engineers to figure out what went wrong and develop a solution. Instead, they wanted to solve the issue with PR: insist that CR somehow ran incorrect or non-stringent testing, have them re-run the tests according to how Apple wants them to be run, and have them revise their recommendation. Obviously I'm extrapolating a bit here, but it feels consistent with Apple's action up till now. Not to mention they put their head of marketing on the case, not any actual engineers. Good on Consumer Reports for sticking to their standards instead of caving to pressure.
    • Re: (Score:3, Insightful)

      by larkost ( 79011 )

      What about the response makes you think that? The only relevant piece of information I actually can see in the response is the inference that Apple is asking them to re-run the tests (presumably with Apple engineers in attendance). The implication is that Apple is trying to reproduce what Consumer Reports saw, and is unable to, so is asking them to do it again. This sounds exactly like what everyone involved should want to happen: make sure that the tests are reproducible, and thus representative of what us

      • by lgw ( 121541 )

        The customer is not your QA lab. CR ran their tests, and saw what they saw. They'll re-test next year. It's not their job to help Apple solve the problems, any more than it is to help Tesla solve their reliability problems (Tesla at least got he Model S off the CR shit-list this year, but it was replaced by the Model X), or a microwave oven vendor, or debug a blender or ...

        • The customer is not your QA lab. CR ran their tests, and saw what they saw.

          Then why haven't they disclosed the full test protocol so that Apple could repeat it? If I test your product, publish a review saying it catches fire, and never mention that my test protocol was throwing it on a fire, are you really to blame? And is my excuse that I'm not your QA lab at all reasonable?

          • Problem with disclosing your full test protocol is it tends to result in producers making their item specially tuned to look good for your protocol--while still being bad in actual use.

            • Problem with disclosing your full test protocol is it tends to result in producers making their item specially tuned to look good for your protocol--while still being bad in actual use.

              OTOH, if you don't disclose it, then it's no more reputable than Rossi's E-Cat or any other "you just have to trust me that it works" test.

              • Unless, of course, you have an solid unbroken 80 year reputation in the testing field, like Consumer Reports does.

  • https://slashdot.org/~TheFakeT... [slashdot.org]

    Shall we wait for him to turn up, or does somebody want to go bait him?

  • "Last week, Apple's VP of Marketing, Phil Schiller insisted that Consumer Reports' findings didn't match the company's field data"

    What's he supposed to say? "Yeah, the whole battery thing is a clusterfuck and Consumer Reports is spot-on."

    • "Last week, Apple's VP of Marketing, Phil Schiller insisted that Consumer Reports' findings didn't match the company's field data"

      What the fuck is "field data"?

  • by mbeckman ( 645148 ) on Friday December 30, 2016 @02:51PM (#53580647)
    Then anyone can run the tests CR refuses to re-run. If they're that confidently of their results, they should be happy to provide the detailed equipment and steps, along with corresponding results, to the public. This is the way science is done: if you make an assertion, then you have to provide the raw data to let someone else try to reproduce your results.

    Anything less is unscientific anecdotal evidence.
    • by guruevi ( 827432 )

      CR is kind of like Gartner and all the other 'benchmark' companies. Whoever subscribes the most to their 'services' gets the top spot. Look at their car lineups, recommending Chevy, Subaru and Ford as "best" in their respective classes. Same for their small electronics department, only recommending big brand names like Linksys, D-Link and Netgear, actually their recommendations are the same routers that have obvious back doors yet they rate them high for security. On the other hand there are a lot of smalle

    • If you disagree with their results, why not devise your own test, perform it, and release your findings? That's just as valid as repeating CR's test, so long as you make a good faith effort to devise a test that faithfully reflects the way users actually use computers.

  • Apple quest to be super thin needs to stop!
    or at the very least not on the mac pro and maybe at least 1 mac mini system.

    The mac pro is held back by being that thin that it can even run both video cards and it's cpu at full power. An bigger one with 2 cpu's will give them the needed pci-e for TB 3.0 or they just to back to the tower case and have an voodoo like look back cable to feed DP over the TB bus like how other pro workstations do it!

  • by 0xdeadbeef ( 28836 ) on Friday December 30, 2016 @03:41PM (#53581013) Homepage Journal

    Since this thread is full of fanboys rationalizing Apple's failures, I think I'll eat their mod points by recounting my personal experiences with their failures.

    I bought a 2007 MBP. It's battery swelled and had to be replaced. Eventually, it's 3d graphics card died and the only way to use it was to boot into safe mode.

    I bought a 2012 MBP. It's trackpad quit working and had to be replaced. The replacement trackpad also failed within a month, but by then it was out of warranty. I quit trying to get it fixed because I use a mouse anyway, and I'm sure those cunts would try to charge me because I didn't buy "Apple Care".

    I was given a 2015 MBP. So far it hasn't failed, but it has behavior that is intolerable. With the lid closed, it goes to sleep unless there is a keyboard plugged in. Apple says "Fuck you, software KVM users". And even with a keyboard plugged in, it immediately goes to sleep if the power cord is yanked out. Apple says, "Fuck you, cat owners".

    I have no interest in their new crippled laptop and its gimmicky function key overlay. That shit was lame when it was called the Optimus Maximus in 2008 and it is just as lame now. Apple says, "But muh innovation! Muh courage!"

    My first laptop, a ThinkPad from 1998, still works and boots to a 2.4 kernel. (Many nostalgia, such rugged, wow.) My other Toshiba, Dell, and HP laptops also worked up until I got rid of them, and they all took way more abuse than my precious, delicate MPBs.

    So this year, I bought a cheap laptop from Dell. I'm using Linux again for the first time in a decade, and it is liberating. Buh-bye Apple, you prissy, shark jumping freaks. I can't wait until I retire and never have to touch your shit again.

    • Re: (Score:3, Insightful)

      by King_TJ ( 85913 )

      0xdeadbeef, you're really just complaining about bad luck that could happen to anyone, regardless of buying Apple products.

      I've been using Apple since around 2001, and owned 6 of their computers at once, at one point in time. I currently work for a company that has deployed about 60 of them to mobile workers and I do support for them (along with another 60 or so using Windows machines).

      The 2007 MBP you're speaking of with the battery that swelled? There were a TON of defective Li-On batteries out there, use

    • by dgatwood ( 11270 )
      For future reference, all Apple repairs carry a 90-day warranty, so if the same part fails again within three months, they'll fix it free. And realistically, they're usually pretty lax about the "same part" bit, so even if something unrelated fails, they'll often treat it as a warranty repair if you ask.
    • by msk ( 6205 )

      Did you try "pmset -a hibernatemode ##" to attempt to control the sleep behavior?

  • I forget now where I read it (might have been over on Engadget)? But supposedly, some employees at Apple spoke about this new Macbook Pro off the record, saying it was supposed to receive a multi-tiered, custom battery in it, similar to what Apple did with the new Macbook in 2015. Except at the last minute, they ran into some issues and were told they'd have to scrap that and just make a standard battery fit inside it instead.

    It wouldn't surprise me a bit if these odd power problems are a direct result. (Had no time to really re-optimize the system for a battery that wasn't going to supply as much power as what they intended all along.)

    I'd have say I side with Consumer Reports on not recommending this notebook right now. I think the touch-bar is very cool and the computer looks great in the new "Space Gray" color option. Not a fan of losing all the ports besides USB-C, *but* if everything else was fine, I'd accept that as a downside I could live with. The problem is, this one seems to have fundamental flaws of the type that you won't see fully corrected until the next revision is released.

    If you've been following things closely on the Mac-specific forums, you'd see there are some serious questions about this computer's video performance too. There's a guy on YouTube who put the high-end configuration through its paces running a number of modern 3D video games and the performance was, frankly, god-awful! In one title, he was only getting 3 or 4FPS! As he admitted himself, people aren't buying the new 15" Macbook Pro as a gaming machine. But they ARE paying a premium price to get the latest AMD Polaris series GPU in it, and that's supposed to be 2 generations newer than the best available mobile GPU AMD had to offer for any older laptops. The graphics performance in games is so abysmally bad though, it's clear something else is going on here. IMO, Apple probably underclocks the GPU to help conserve power and to control heat generation -- and may have done so far too aggressively, given the last minute battery change that had to be done.

    • by dgatwood ( 11270 )
      It's basically a revision one product. Long-time Apple fans know what that means.... Better to wait for the second rev. :-)
      • by skaag ( 206358 )

        As a long time Mac user, this is exactly what I'm doing. It's not only over-priced, but I truly see no advantage to upgrading from my current Macbook Pro to the new one. If anything, I'd have to buy tons of adapters which I'm not keen on doing, and until GPU performance is fixed on laptops, there's no way I'm going to upgrade.

Avoid strange women and temporary variables.

Working...