I was called out today to look at a Windows 2012 R2 server that has been running happily since it was installed back in June of this year. When it was installed, we also installed a Synology DS415+ NAS and 2 x 4TB drives which were mirrored to provide data protection. The available space on the NAS was split into 2 volumes. One volume was used for files that were needed locally / remotely and the other volume was used to present an iSCSI volume to the server as a local Hard Disk so that the backups were written onto the NAS.
Earlier in the month (7th October) I received several Improper Shutdown emails from the NAS as the power supply had fallen out of the back of the NAS because someone had knocked the power adapter that comes with it and that had caused the plug (4-pin DIN plug) to come out of the back of the NAS.
After the Improper Shutdowns, everything had been fine, backups were working to the NAS and all was well.
Around 3:00pm on the 15th October, our office received a call advising us that the server had Blue-Screened with a BAD_POOL_CALLER blue screen of death and could we help to get it working again. Numerous options were tried by our support staff to get the server back up and running again but all proved fruitless and so I was asked to attend site 1st thing in the morning.
When I arrived, I noticed that the NAS was not working properly (the power light was on but no LAN lights), so I switched it off. I then went to look at the server which had been left in Safe Mode overnight. Digging around I couldn’t see anything screamingly obvious as to why the server had suddenly died. There had been no Windows Updates, no system changes, no power issues, nothing that I found using Google seemed to fit the issue I was having, so I gave the server a reboot and randomly it rebooted cleanly. I logged in to the server and it seemed totally happy.
After leaving it running for a few minutes, I decided that if that server was fine, the NAS should be switched back on again so the users could access the files on the NAS. After switching on the NAS, I sat down to drink a cup of tea and discuss how weird it was that the server booted happily after failing to do so numerous times the night before. A few minutes later, the server fans went into turbo mode and I got that sinking feeling that the server had crashed again and it indeed had.
So – I switched off the NAS, rebooted the server, which booted happily, removed the iSCSI connections to the NAS, powered on the NAS and everything was working happily again. At some point in the very near future, I will zap the volume, zap the iSCSI LUN / Target on the NAS, re-create them and then re-connect the server back up and hope that the problem goes away.
So – if you receive a BAD_POOL_CALLER Blue Screen of Death error on a Windows Server using an iSCSI volume on a NAS, then it may well be the iSCSI volume that caused the problem and you will hopefully know how to fix it.
Filed under: Blue Screen Of Death, iSCSI, Synology NAS, Windows Server 2012 R2 | Tagged: BAD_POOL_CALLER, BSOD, iSCSI, Synology NAS, Windows 2012 R2 |
Alan Hardisty's Blog - All Things IT Related 
Did you resolve? I had 3 servers that crashed on October 15 all 3 had iScisi conenctions to NAS devices and when the NAS are OFF… no problem…. All 3 NAS are different models…. No Windows updates were isntalled on any of the 3 servers in the last 2 weeks….
Not as yet – the solution to the problem has been left with the iSCSI drives disconnected. Too much down-time and not wanting to upset the customer more has meant I’ll leave this for a while before trying again.
Wondering if there was an update to the OS of the NAS – which I haven’t checked as yet.
Alan
We haven’t found consitent cause either regarding this issue We are also trying to avoid too much downtime for our clients. Same for now, NAS remain OFFLINE.. Please update if you find more information.
I’ve deleted the iSCSI drive on the NAS and left the server and NAS on and working, just not talking to each other for now. This particular customer has data on the NAS, so leaving it off wasn’t a solution for them. Are you using a Synology NAS too?
So you managed to give them access to the data even without ISCSI?
All 3 servers are using 3 different models of NAS but they all crashed on same day, one is a Buffalo, one is EMC and one is QNAP.
Yes – but the data on the iSCSI partition was only being used for server backups, not data files. I disconnected all the iSCSI settings via Control Panel / iSCSI Initiator and deleted / re-fromatted the iSCSI partition on the NAS and left them disconnected from each other. Interesting that all 3 of yours went down on the same day and them all being different manufacturers. Suggests that the problem lies on the Windows side of things to me.
Hey Alan, what antivirus does the problematic server use on your end? Bitdefender for us..
Well what a surprise – BitDefender is also on my problem server. So maybe an update to the definitions has caused this?
Very possible…. 🙂 Let’s keep in touch!
Absolutely. I’ll call / contact BitDefender tomorrow and see if they have any ideas. We have about 900 endpoints using BitDefender, so might have a better route to support. Did you struggle to find any info about this issue before you found my post?
Yeah I couldn’t find anything regarding the issue on Microsoft side, so I starting assuming nothing was wrong on that end and looked elsewhere… For some reason we also have other client with BD and NAS that have had no problem though….. So I ended up uninstalling BD and updating my NAS firmwares at one of the problematic client since they were ok with small downtime and so far no problem with that one… So I will try to reinstall BD now and see how it goes…!
Could you kindly edit my last post and remove my signature, it got added to the post automatically since I replied from Outlook.
Thank you
What signature? 😉
Thanks
Did you get an anser and any useful information from Bitdefender? I uninstalled it from all 3 servers and I was able to get my all NAS back online without any crash… I’m wondering if there is something we canput in place in the BD endpont policy for those servers to avoid further issues…?
Nothing back from them yet. Have you setup the Network part of the Firewall to include their internal IP Range?
No I dont beleive so..
Presumably your NAS is on the same IP range as your server? If it is – make sure you define the internal IP range as that may be interfering with the NAS communications.
I will look into that..
Whats strange is that it had been working just fine until then for a long long time… something must have changed in Bitdefender but somehow we also have other clients with BD and NAS and same policy and no problem….
Yep – that is very odd.