Issue with my latest Pulse XE
I have been running printers now 24/7 for 6 months and have started running production parts a few weeks ago as I am now confident in the process and end product.
I also run a software and app development company for GPS systems...and while I don't sit down and write the code... I pride myself in my bug hunting abilities and being able to pattern those bugs for the developers to kill... just putting that out there so those listening know, that I'm not just throwing this out there as "user error" and I have done a lot this week to try and pattern and remove any and all factors that may be causing this external of the printer itself. (Side Note: I have been working hard to pattern my last reported bug with the Matter Control Slicer Export...being stuck open...which is still happening)
I currently have my original Pulse XE (model E-443S), my newer Pulse XE (the problem child/model E444M), Prusa Mk3 and a Voron Trident...all dialed in perfect and running day and night except...
The newer Pulse XE (model E444M)
Running both my Pulse XE's and my Prusa Mk3 via cable to a iMac desktop running MatterControl to slice and run the parts.
Everything is updated.
Almost every second or third print job (average 2-4 hour prints) the part will be printing absolutely flawless. Then all of a sudden it will be like the printer is no longer climbing in the Z...best way I can describe it and you can see in the attached photos and videos. UPDATE to this. I added a photo of my last failure that I let run out a bit and it almost looks like is switching to traveling DOWN in the Z as you can see its actually gouging out the part
This is NOT a Hardware issue or mechanical this is either in the software of the printer or an issue it has in the connection with the computer...but I believe this is internal to the printer as this printer has had a few other quirks and does not act the same as my original Pulse XE. (They are both 32 bit but my original Pulse XE has the larger controller and screen versus the smaller one I now have on this newer model)
I know this because every time it happens I get the same result on the computer running MatterControl. When I go to stop the print, the toolhead just sits there on the part and the MatterControl Print Queue popup becomes unresponsive. After a few seconds an Error Popup will appear on the screen (See Video labeled "WATCH THIS" for the full repeatable pattern).
Those other quirks with this machine have similar to where if the print is doing fine but off to a bad bed start, and I cancel it...it will lock up and I have to power down and up the Printer to Home it off the part.
Things I have done
- Off of MatterHackers support recommendations, I deleted the Printer Profile and Added it again. I had high hopes for this as they said it might have gotten corrupted and setting up a new profile may fix it. Also I noticed when I did it I had a few more Slicer options so I assumed the Profile for this model may have been updated.
This did not work as I am still seeing the same print failure and pattern.
- I switched all my printers to another computer (MacBook Pro M1) and set up new profiles
This did not work as I am still seeing the same print failure and pattern
I am printing in PC+CF and some other filaments up around $5 a part...so this printers failed prints...and hour or two in is getting expensive!
In the dropbox folder
- see the sample part photos where it just all of a sudden starts printing onto itself
- see the WATCH THIS w/ ERROR video for the full pattern with the popup error I get when this happens (these just happen to be smaller parts and harder to see as I caught it early, so look at my larger part video and pics to see the result if left unattended)
- I attached the GCODE txt file of one of the failed prints right up to the time I powered down the printer after the Error popup. You can go all the way down to the end to see how the printer was running up until it failed. Maybe this will give a clue...
I am running currently in production mode and have been testing and running printers from these desktops to industrial Fortus, Essentium... even over those high priced options I am happy with what I am getting in ROE on these and how well I have them producing functional parts.
While the VORON Trident, I just finished building, may be the large footprint 350x350 winner of the 3, with its performance, higher temp and 60c chamber... out of the Pulse and Prusa...I am very happy with my first and original Pulse XE. Even without it being direct drive...I have figured out the large retractions needed to print even better looking parts over my Prusa. Rather than start adding Prusa's I was looking to add more Pulse XE's but my second go at it with this unit is not going so great...which is a shame because before the "crash"...it is printing just as beautiful as the first Pulse I owned.
Hope to get this issue figured out!
@DamianOfftheGrid Thanks for the detailed analysis. Taylor ( @cope413 ) and I will look into these reports and get you sorted. We have not seen this error before but it does look like it might be something wrong with the firmware or controller.
We will look into what is the likely cause and Taylor will reach out to you to either try a few things (like new firmware) or we will have you ship it back and get a new machine out to you while we figure out what is up with this one.
Thanks for all the investigation.
It looks like your Z motor drivers are overheating and dropping current until they don't have enough power to move up.
I'm not sure how/why, but your starting Z motor current is showing at 850ma, which is higher than the firmware default (725ma).
So, the first thing we should do is re-flash your firmware and confirm your drivers are starting at the right current.
I'll email you the firmware with instructions to flash it and we'll go from there.
We can actually see the error in the terminal log that you exported (thanks for that).
00:00: Z driver overtemperature warning! (850mA)
It appears to me he is also printing in an enclosed space which may be adding to the over heated motors. @60C I would experience numerous errors, even with forced air cooling of my control board and drivers.
@tinken Those cabinets are only getting to 34c with the Pulse's
@cope413 Received your email and I am going to troubleshoot based off what you sent.
This isn't a heat or "constant" issue for those listening in. Both Pulses will run beautiful 3-4 hour...sometimes 12 hour prints, flawlessly as I run these 24/7. Cabinets stay at a constant 34c.
This particular Pulse included, but every 2-3 print jobs it will "all of a sudden" (and I mean within an instant) go to crap and start destroying the print. This is not a "gradual thing". At the same time Matter Control will lock up on the print job when you try and cancel the print.
This is not a climbing heat issue...as this will happen after multiple prints with the cabinet maintaining that temp. Again its an instant crash in the print job and the photos I attached are after only letting 2-3 layers print after the crash. So what you are seeing is not more than a few minutes (3-5 minutes) once I see it go bad.
I know the gantry is level, will confirm the bed but I am pretty sure its level as I print parts all around it with a consistent "squish" on the first layer.
Will verify the steppers as suggested.
Again, I ran and still run the original Pulse XE in that same cabinet at those same temps and have done so for almost 6 months with a hundred rolls of filament through it... this new Pulse XE out of the box, just acted a little "different"...
I don't know what would cause the issue, but it's very clear what the issue is. In your print log that you shared you can see that the current for the Z starts at 850ma when it throws the first over-temp warning (line 25272) and drops over the course of the print all the way down to 60ma (line 199842).
It's odd for a few reasons, but first and foremost in my mind is that the default current for the Z motors has been 725ma for a long time - so I'm not sure how/why yours is showing 850 to start.
At 34C ambient, you're right, there shouldn't be any over-temp issue, but the driver temperature and subsequent current reduction that the firmware does when over-temp is detected is definitely the cause of the issue.
I would expect it to be a fairly sudden failure mode, though, because at a certain point the motors simply will not have enough power to move up. I would expect it to be a very binary failure mode. It will work until it suddenly doesn't - I just don't know what that current value is because there are a lot of variables (friction of your specific machine, specific impedance of the motors on your machine, specific voltage of your power supply, etc)
One thing I didn't mention in the email is that you should send an M502 command after flashing the firmware. This will reset the EEPROM to the firmware default values.
@cope413 so off your emails… you sounded like in the second, that you saw the GCode and thought it may be the stepper drivers.
Confirmed the gantry and bed is level/flat.
Should I just go through that process of trouble shooting the drivers? Or should I mess with the firmware still, prior to that?
Just confirming…because I’d like to troubleshoot in increments versus throwing the kitchen sink at it all at once.
Sounds like from this last post you seem to know the issue…based off the GCode.
Regardless it sounds like this printer is turning out to be a “little off”…
Sorry if I wasn't clear. I meant to convey that we definitely know the mechanism of the failure - that is, motor current dropped low enough so that the Z motors were not able to move anymore.
I just don't know the exact cause of that failure. Said differently, assuming that your ambient is ~34C, and that you haven't manually changed the motor currents, I don't know what would cause your drivers to throw the over-temp error and start dropping current. I also don't know how your drivers were set to 850 to start.
Let me know if the failure mode occurs after flashing the firmware.
@cope413 Running tests now with the firmware updated.
Other than the M502, is there a thermistor calibration I can/should run?
It is now taking a while to hit the nozzle temp and what I am seeing is (with nozzle set to 263) that it climbs to 261/262 and flutters back and fourth between the two for what has been at least 5 minutes or more on both tests so far.
I know on Klipper I can run a thermistor calibration...
This was not like this before, so I am assuming it lost something in the firmware update.
*In fact while I write this it has been at least 8-10 minutes of it fluttering back and fourth between 261/262 without yet hitting the 263 to start the print....going to have to cancel it or adjust the temp down to kick it off...
Again this printer acts in no way like my original Pulse XE....like a completely different machine.
@cope413 Yea so the only way I can get it to print now, is to set the nozzle in the slice to a specific temp (higher than I need), then start the print and when it gets stuck fluttering back and fourth under that set temp (I have let this go on for 10 minutes at least in one test) I then need to manually bump the temp down in the Control Panel to get it to start printing....
Here is the Terminal log of it fluttering under the set temp
Temp not reached log
@damianoffthegrid Run a PID autotune.
Send this command in the terminal
M303 E0 S275
When it's finished, it will spit out 3 values, (Kp, Ki, Kd)
Write those down and enter them in the EEPROM.
@cope413 Ok ran it...but sorry...where and how do I access this "EEPROM"... NEVERMIND I found it....
@cope413 Still getting a Hardware error on heat up.
Error reported: "Error-Heating failed, system stopped! Heater_ID:E0"
and here is the exported log
@cope413 Just getting repeats of those error logs and cannot kick off a print. Here is the last one where it started to do its bed leveling and "heated out".
I was only able to get one print out of it after the firmware update today and that was earlier prior to PID autotune and that was by chance as I quickly lowered the control panel nozzle temp after the nozzle was warming up past that temp. (FYI the print was beautiful...but have tried nearly a dozen times and cannot get it to start another print without that error)
The Print in
PC+CF .15 layer height 40mmx20mm Cube
*and that's a first print, new profile and after pulling the printer off the shelf, re tightening everything, re leveling.... 6 months of R&D...."dialed"
I'm gonna give this "Lemon" another night...then its getting pulled off the shelf and this Prusa Mk3 sitting here is going up in its place. Unfortunately, while I like the first Pulse XE, and this one prints great...when it prints....I got parts to run and this thing ain't cutting it.
Let me know any last suggestions before it comes back to you.
NOTE: If I had to guess...the firmware you sent does not have the appropriate thermistor type w/ mosquito hotend...as it seems like the nozzle is now taking twice as long to heat up n and the bed is beating it to 90c
I don't ever remember a time...when the bed would heat up faster to 90c than a nozzle would get to 260c....
And just an Update.... I will confirm this right now.
The original Pulse XE on Preheat, will get to 280c at the nozzle while the bed is still halfway to 90c
The "Lemon" Pulse XE will get to 90c on the bed and sometimes even start bed leveling before the nozzle even gets to 240c (goal 260c).
Right when the Bed hits 90c it errors out because the nozzle didn't beat it to temp.
That's your issue....
This post is deleted!
@cope413 and here is some more great findings and videos of these findings....
So with this above information I attempted a Preheat with the bed down low so that it would never cool down to that temp while allowing the hotend to heat up.
I had it originally set to 260. It climbed then flustered around 240/241....stayed there.
I then turned it up to 280. It then started to kick in and climb again till it stayed and fluttered around 260/261.
Here is that video
I then set it to 300. It took a few minutes... then started to climb to 280...
Here is that video
Hopefully all that helps... I am about done toying with it....
@damianoffthegrid "The Print in
PC+CF .15 layer height 40mmx20mm Cube" Just an fyi, if you are printing composite polycarbonate, you should never exceed a layer limit of 60% of your nozzle diameter. So if you are printing with a 0.4mm nozzle, the layer height should not be lower than 0.24mm. Layers lower than this limit can and will eventually cause excessive pressure and clogging.
@tinken it’s a .6 nozzle on that cube/all my production parts and machines and I run PC+CF all day n night at .15 layer height…. Prints beautifully as you can see…hours on end on complex parts with tons of retractions… 9mm at 85 mm/s to be exact on every travel move…
But this really has nothing to do with the thread….or the current issue with this printer that we are discussing