Tuesday, October 30, 2007

Classroom Spy in the enterprise?

We have long been searching for a piece of software that will let multiple users in our network view over the LAN what's running on one specific computer. This will save us licensing fees to run the same software (Flight Explorer) on all workstations. After a long search of various scenarios - split video signals going to multiple large-screen wall-mount monitors - we found and have decided upon a most unusual solution. We found a piece of software called Classroom Spy Professional. One of its many features is that it allows a master computer - the teacher - to push a video signal out to dozens of other computers over the LAN. The users can view that screen in a small window or on a second monitor.

This seems like a most unlikely piece of software to be used in a professional enterprise environment. Why? Because it is classified as spyware. I had to add an exception to our global anti-virus rules in Symantec AV Corporate. In fact this is the third piece of software I have had to add to the exceptions list. We use another piece of software called Track4Win to monitor the web sites our employees visit. The third piece is IPScan from Angry Ziber. It is a great little network tool that quickly allows the network admin to see all IP addresses in use in a subnet.

I wonder why Symantec classifies as spyware three great tools that help me do my job.

Saturday, October 27, 2007

Does everybody really need their own printer?

I was travelling most of the day yesterday to visit family out of state for the weekend. While driving I carried on an ongoing email dialog on my Treo with an associate who was helping to troubleshoot a printer issue in one of our offices with a large open floor of employees. We have two network printers there and you would think that the employees in that office would have no problem getting up from their desks and walking a few feet to the printer when they need their documents.

As he was carrying a replacement printer to the office he was stopped by employees in another department asking if he had any spare personal printers that they could use in their office. Now these employees also have a couple of network printers less than ten steps from any desk in the office. What is it with these people? Is it so hard to get up to get your printout? At what point is a personal printer justified in a small office environment?

Of course the right answer to these kinds of request is, "Of course you may have your own personal printer. Just shoot me an email request and I'll get your boss to sign off on it. Once she approves it we'll forward to the CEO for final authorization." Almost always the employee will respond, "never mind." The same response works well for requests for larger monitors - do you really need a 23" monitor - and for a yet another faster computer.

Managing those pesky users is a major part of the job for an IT Manager.

Thursday, October 25, 2007

Is prepaid tech support really necessary?

We run three mobile email servers - Exchange with ActiveSync, Goodlink from Good Technology (now Motorola) and BlackBerry from RIM. Only BlackBerry wants us to pay in advance each year for tech support that we have never used. It's not cheap either.

I was amused when RIM sent a renewal invoice for the annual tech support. The prices were in English pounds sterling. So I emailed back and asked, "how much is that in US Dollars?" They sent me an email thanking them for contacting tech support and promising to get back to me within 24 hours.

Several days later I receive another email from RIM saying that they have provided me with a new quote in US Dollars. I email back and ask, "so where is it?" They had forgotten to attach it. They email back thanking me once again for contacting tech support. Several days later I receive another email asking if I'm going to renew our annual tech support.

By now I'm getting tired of this. I email back informing them that they neglected to send me a quote that I can use. They thank me for contacting tech support. I finally get a quote in another email and discuss it with management. We decide that it is not worth the $1,200 fee for something we haven't used in the past year. It smacks of 'protection money'.

What do you think? Is prepaid tech support worth it? I figure if the server ever goes down I can switch the BlackBerry users to a Smart Phone or reinstall the server software. Maybe if we had hundreds or thousands of BlackBerry users I would feel differently but we only have a dozen. Almost all our mobile email users have switched to BlackJacks with ActiveSync.

Tuesday, October 23, 2007

I just love RAID and hot-swap drives

One of the drives on our Exchange Server went bad over the weekend. Luckily I had a spare on hand and was able to get the array rebuilt right away. That’s the good thing about RAID. You can lose a drive and still keep running. That’s also the good thing about hot swap drives. You can replace one with the power on and nobody notices the server came that close to suffering a catastrophic meltdown. I just hope I can still get a replacement for the old 36GB hot swap drive so I can have a spare on hand.

I would never run a major server like an Exchange server or a SQL server without RAID and hot-swap drives. We also run redundant and hot-swap power supplies and fans on those critical servers. We exclusively use HP Proliant servers and have been for many years. Yes, it costs a little bit more than your no-name clone but I'm not about to trust our critical email and accounting systems to anything less reliable. I suppose if I worked in a larger shop I would find Dells but in every small business I worked at for the past twelve years, we've used HP servers.

Wednesday, October 17, 2007

Techie in a non-tech environment

I read a great article on Dice this morning. For those who don't know, Dice is the premier site for guys like me who want to keep their finger on the pulse of what skills are in demand. It's a job board first, but provides a lot more than that. This article is an example of why I still visit Dice everyday even though I'm not actively seeking employment. I even have Dice send me daily emails of new jobs posted in my neck of the woods.

Times must be good because on some days there are a dozen or more new listings within ten miles of the small little town of Camarillo CA where I live. Some listings are obvious attempts to suck in new applicants to fill the pool of the new headhunter. Most are attempts to find someone with that obscure skill that most tech guys like me will probably never have.

The writer of the article describes his life as a techie in a non-tech world. I was interested because that's exactly how I spend my day as the IT Manager for a very non-tech company. I was surprised by his take on the whole situation. Instead of focusing on the benefits of being the top tech dog, he described why it was not for him. Apparently he missed the feedback from other techies that he used to get in his previous job. I've gone through that too but as I get older I realize that the perks of such a position far outweigh the downside that he writes about.

For example, I just love going into a meeting to buy new hardware because usually the boss has already been convinced by other managers that the purchase is necessary. I may not be up on all the latest buzzwords and tech offerings but I do feel like a major contributor to the success of the company. While it's true that the boss usually has no interest in IT stuff, when it becomes obvious that the lack of his participation in the process will affect his ability to do business, he becomes very attentive.

The writer has a point that other department managers can be very naive and mistaken when it comes to what you can and can't do as an IT Manager. Yep, they tend to think that you are in charge of anything that uses electricity or that you talk into, but that's OK. I don't mind being the jack of all trades, especially since I use outside consultants for the heavy lifting when it comes to some very specialized technology. I guess it all depends on the company structure.

Working with a lot of other techno geeks is good when getting started in a career but eventually you need to stand on your own. It requires a little more research and digging to find exactly the right solution but that suits me just fine. There are a lot of people out there who could do the tech side on my job better than I do, but my contribution to problem solving and finding new ways to utilize one of our greatest assets - the flow of information - is richly rewarding.

The new server is installed!

We installed the new server yesterday. It has new disk drive technology on it that I have not seen or used before. The drives are very small - the 2.5" size that are used in laptops. They are still hot-swap so they cost an arm and a leg more. We set up the OS and the SQL Server transaction logs on their own mirrored sets on the server itself. For the data we had to go to an external storage enclosure which holds regular sized hot swap drives. The data drives are set up on RAID 1+0 - mirrored sets of a striped set. So even though the drives are 300GB each and we have eight of them, we only have 1.2TB of usable space. We split that into two logical drives of 558GB each - NTFS overhead takes up the rest.

Monday, October 15, 2007

The new rack is installed!

You know I must be a major geek when I get excited about a new server rack. But here it is. Isn't it a beauty? It's the empty black one on the left - an APC Netshelter. As you can see our old rack was not a standard size rack. The new rack is taller and deeper. The server, external storage enclosure, UPS and auxiliary battery unit are all in boxes in the server room today ready to be installed tomorrow. We have a new HP TFT7600 slide-out rack mount keyboard & monitor to take the place of the old desktop units that look so out of place in the old rack.

That's an Avocent 8-port KVM switch under the three HP Proliant servers in the old rack. We will continue to use it on the old rack since six servers are still on that side. You may be wondering what that aluminum duct is behind the new rack. We have an auxiliary portable air conditioner to help the underpowered wall unit we put in the server room last year. The wall unit was a disappointment. It was supposed to replace the portable AC but never did put out the advertised BTUs so we continued to use them both.

The old SQL server is the top unit in the old rack. It is an HP Proliant DL380 G3 - Dual Xeon 3.1Ghz processors, 1GB memory, 256GB RAID 5 storage. It is the most overloaded server I have ever worked with. I would never load it up this way. It was set up before I came on board. Not only is it running SQL Server 2000, it is also our master domain controller for Active Directory as well as our file and print server, DNS and WINS. It is an Application Server hosting our accounting system, our payroll system and our backup system - Symantec Backup Exec 11d with the LTO 3 tape attached. It runs our internet tracking database (yes, we automatically track everywhere our employees go) and our Jetnet database of available aircraft for sale.

I have tried to reduce the load by offloading DHCP, RRAS, SMTP Gateway, Anti-Virus and lots of other applications to other servers over the years. Talk about single point of failure. I wonder if my predecessor really understood how severs should be setup. Either he thought one server could handle everything or maybe he just didn't know how to get additional servers funded. This is the third new server I have added in the last two years. Next year I hope to put in new server hardware to support Exchange Server 2007.

Sunday, October 14, 2007

Got Spam? Try Commtouch

When I came on board with my current company, we had a serious problem with spam that was only getting worse. My predecessor was trying to manage it with a Symantec product that was not designed to control spam but to keep viruses out. He was frantically trying to block subjects and addresses in a never-ending battle.

Some of the executives were so fed up with the onslaught of spam that they purchased and installed Cloudmark Safety Bar. That's not a cost effective solution for a small business. While it helped it did not block the spam before it got to them and they still had to see it before Cloudmark did its thing. I immediately researched the available products and services to find something better.

After much research I decided on Commtouch, which a lot of vendors I talk to still have not heard of even after several years. It seems to be a very well kept secret. I love the product and associated service. It does a wonderful job. Spam for the top email recipients dropped from hundreds of pieces of spam each day to maybe one or two. For most users it dropped from dozens to one or two a week. I think we pay less than $30 per user per year. It is a bargain.

It is also easy to manage and administer. The users can manage their own daily reports of what was blocked. They can either review the report or tell the system to stop sending it. Some want the report, others don't. It even has an Outlook plug-in but I recommend they don't use it because it makes the product perform just like Cloudmark. Why would they even want to see the spam? I say block it before it gets to your mailbox and cancel the daily report. The product is so accurate that I have literally had only one false positive in over two years.

The long-time employees respond that the product does miracles compared to the crap with which they used to have to put up. Unfortunately, the newer employees are spoiled. No anti-spam solution is 100% accurate, just like no anti-virus protection is perfect. We can go for weeks with no spam and then an outbreak will occur. The spammers hit upon some new method and it takes the blocking engine a few seconds to learn and block the mutation. In the meantime a few slip through. You would think a catastrophe had occurred the way some of the new employees respond. "Why, how could this happen? We've never seen this before!"

I feel like saying things like, "Get a life kids. Grow up. Find something else to complain about. Do your job and stop trying to do mine." But I would never say things like that. I just smile and email back, "I'm truly sorry you were inconvenienced by the single piece of spam that got through to your mailbox. Please accept my apologies for the integrity breach. I will inform the managers of the spam blocking system right away to make sure this never happens again."

At first they don't get it. Come on - spam happens. We get nearly a half million pieces of email a month. We have a 99.997% blocking rate. That's 50 pieces a day that still get through. Most of those go to the long-time employees but you are bound to eventually get one of them. Sheesh! I'm sure there are more expensive anti-spam solutions but I highly recommend Commtouch.

Saturday, October 13, 2007

Why does the DSL go out so often?

We have redundant internet connections at the office. The T1 is more reliable and I use it to keep our four remote locations connected via permanent VPNs. It is also our SMTP gateway and primary portal for incoming VPNs via RRAS. Our T1 service is fairly inexpensive - less than $300 a month for a full 1.5Mbs up and down from Speakeasy. It's the uplink speed that is important to us to get our email through the pipe as quickly as possible. We send a lot of large attachments - mainly photos of aircraft for sale.

In order to keep the T1 free for serious internet traffic (email and VPNs) I got an inexpensive ($60/month) DSL from DSL Extreme. It is 3Mbs down and 768K up. We don't need the uplink speed on the DSL but the users appreciate the quick downlink for their web searches. We do more and more business through the web these days - links to FAA sites and such. Unfortunately the DSL is not very reliable. It seems to go out every few days. It can be maddening. Sometimes it will work fine for a week and then it will fail two or three times in one day.

Today was one of those days. I few months ago I got tired of having to drive fifty miles into the office on a Saturday just to reset the DSL. The Saturday staff can't reset it for me because the server room is in a locked area behind the accounting office which is also locked. So I bought something called a PowerPal from DataProbe. It is a little $225 remote controlled power switch. It requires a phone line to access the remote on-off capabilities. I chose to have it on a dedicated line but you can piggyback on a FAX or modem line.

It is really simple to use. You just call the number and press a certain key in between the first and second ring. It can be programmed with a security code but I have never found it necessary. Once it hears the keypress it responds with a tone indicating if it is off or on. You then press another key and it does a 5-second power cycle, with a tone when it is back on. I have my DSL modem plugged into the PowerPal and so far, it has worked every time I have had to use it. It has saved me many trips into the office.

My only question is, why does the stupid DSL line go out so often? I have a similar problem on my DSL line at home through Verizon. It can go for months without any disconnects and then will experience outages every few days for a week or two. It's as if the ISP is reprogramming or resetting it on their end which somehow drops the signal on our end until the modem power is cycled. I don't know much about DSLAMs but you would think they have progressed to the point that someone in the local loop could be added or changed without messing everything up.

Wednesday, October 10, 2007

You would think they would have learned

How many years have viruses and trojans been around? Ten at least, right? Maybe fifteen. Haven't most users at the executive level been computing about that long? You would think so. Then why is it that the users at that level are the most gullible when it comes to opening email that is questionable? Maybe it's because they get so much email that they just don't have time to think before opening - just open and look. Hmmm...nothing happened. I know, I'll forward it to the HR manager since it has the word resume in it. Great, another manager too busy to think. Why open an attachment from someone who said they saw our ad on Craigslist. Hey, we're not even advertising on Craigslist right now. And it's a zip file! Doh!

My whole day was shot today tracking down and eradicating this nasty trojan called Proxy-Agent.aj or Spam-Mailbot. The offending file is wmupdate.exe found in System32 but figuring that out took eight hours and a half dozen scans from every available anti-virus publisher. We run Symantec AV Corporate edition which includes protection at the SMTP gateway, on the Exchange server and at the desktop. But believe it or not, Symantec doesn't recognize this one. We pay big bucks for their protection. This one has been out since Dec 2005. You would think they would have it in their database by now, but no. The symptoms are continual pop-ups from Symantec Email Proxy saying, "Your email message to ... with the subject of ... was unable to be sent. Your email server rejected the message." Hundreds of the little buggers as long as you're connected to the internet.

I ran a full SAV scan on the workstation. It reports no malware. I download, install and run a full system scan of AVGFree. It finds nothing. A fresh update and scan of Spybot - also nothing. The same thing for Adaware - nothing. I know I'm not crazy. The pop-ups continue when I reconnect the workstation to the internet. Somebody has got to know about this. I know. I'll try TrendMicro House call. That has always worked in the past - nothing. The pop-ups continue. TrendMicro Sysclean, Sophos Anti-rootkit, CA eTrust - nothing. Ah, I forgot McAfee. Let's try that. They have a free online scan. Wow! It detected something, but won't remove it unless I fork over $39.95. Tough. They identified it so I just deleted it. It worked but I've wasted eight hours. I'm not happy with Symantec. Why didn't their products detect it?

Here is a link to more info on Experts Exchange. I found it after the fact by Googling resume.zip craigslist. I wish I had thought to Google that at the beginning of the day. Live and learn.

Update: Symantec has a web page where you can advise them of new variants of viruses. That's apparently what we got. I guess someone has to be the first to get it, eradicate it and then advise them. These viruses mutate so rapidly that not even big companies like Symantec with all their resources can stay on top of it. Our Symantec reseller recommends we implement an Intrusion Prevention System (IPS). The better IPS products not only protect against well known attacks via signatures but can also detect and block previously unknown attacks such as protocol anomalies (non-RFC compliant protocol traffic).

Monday, October 8, 2007

A new server project was approved today!

The boss approved the purchase of a new server and rack today. We've outgrown our old SQL server and are moving up to a 1.2TB RAID 10 unit. It's an HP Proliant DL380 G5 Quad Core with 4GB of memory. We've been using RAID 5 for many years. The new RAID 10 should give us a major boost in performance for our new document management system that will reside on the new server next year. It requires an external storage enclosure for the RAID because there are 12 drives. That along with the UPS and second battery puts us at 9 rack units and I only have 8 available on the old rack. I'm also going to get a nice slide-out keyboard and monitor in the new rack which should provide lots of expansion for years to come.

It wasn't as hard a sell as I thought it would be. The CFO, Controller and I had done a lot of research into document management systems and had a lot of ammunition for the review meeting. But all the CEO wanted to know was why we needed it, why we needed it now and what would happen if we didn't buy it now. We are running SQL Server 2000 and support for that product expires in January of 2008 so we are going to SQL Server 2005. Our old SQL Server is running on hardware that is no longer in warranty and coincidentally, one of the drives in the array failed just this morning. We are getting server 2003 with software assurance so we can upgrade to server 2008 next year when it comes out. If only I had software assurance for my other 10 servers. That's going to be a big capital expenditure next year along with Vista on all the workstations, Office 2007 and Exchange Server 2007. I'm not convinced we need all that just yet. There just aren't enough compelling business advantages to drive the change.

Thursday, October 4, 2007

External logo for Outlook Signature blocks

This is real geeky stuff but it's what I do all day. We have had an ongoing problem with logos in Outlook Signatures being broken or missing. This is especially a problem for the executive staff who do lots of email from home. We'll set up their signature block to point to a local copy or a server copy of our logo.

Then somehow that logo will get deleted or moved or a mapped drive will fail to map on logon or something else will cause it to not work. There's nothing more unattractive and unprofessional than receiving an email from the CEO with a big red X where the company logo should be. In fact it can be downright embarrassing when trying to impress a new client.

So we decided to point the signature block address to a copy of our logo stored on our external web server. Then we just make sure that we have set up the signature properly on whatever machine the employee uses to access Outlook email - at work or at home or both. It even works great for our road warriors who live out of their laptops.

It really is quite simple. The signature block is simple HTML code. We don't like to create the signature directly in Outlook because it bloats the file and creates subfolders for the logo - or rather for a copy of the logo. So we create a lean and mean piece of code with an embedded link to the logo on the web server and put it in the right folder on the workstations.

By the way, that folder location is Docs & Settings \ Username \ Application Data \ Microsoft \ Signatures. Once you place the HTML file in the folder and turn on the signature block from within Outlook it automatically creates the .txt and .rtf versions of the file that it requires. Of course Outlook must be set to use HTML format when creating or responding to emails in order for the logo to show up.

Here is picture of the sample code that worked for us.

Tuesday, October 2, 2007

Is Rev Control really necessary in a small company?

We don't do much software development at my company. In fact, I would have to say that we don't do any at all. We use off-the-shelf packages that have been slightly customized by the VAR that sold them to us. When we need something modified, we contact them to update the SQL code or an occasional Crystal Report. I can do both, but I prefer to let the reseller take care of it in case there are dependencies that I might not know about.

Our web site is about the only thing we modify on a regular basis ourselves. We didn't design it but we sure do maintain it. Some weeks there are dozens of changes. Some weeks there are none. Most of the changes are content related - new aircraft listed for sale, new employment opportunities, an occasional aircraft added to our charter fleet - stuff like that. There are two of us that work on the website - me and the General Manager's son, who I am training.

When a user requests a change to the website they usually send us an email with the new content. If they try to do a verbal I insist they put in in writing so there are no assumptions or misunderstandings. Usually they send the changes to me and I decide if I will do the work or assign it to my associate. We have one user who insists on sending the request to both of us. I guess he figures his chances are better that it will get the attention he wants that way.

Unfortunately, this can cause some problems. I have trained my assistant to be responsive to requests like this since we are a service department and don't produce revenue. In other words, the employees are our customers. I work from home several days a week so I am rarely in the office at the same time as my co-worker. He is part-time so I schedule my time on-site to be when he is not there. This provides the best coverage for support issues.

I made a change to one section of the website last week and my associate made a change to the same area this week. Unfortunately, we duplicated effort - his post was added to mine and we ended up with the same entry twice with slightly different wording and layout. He was just being responsive to the user request and didn't realize that I had already done so. I suppose we could say it is the user's fault for sending the request to both of us but I would never do that.

My point is that even in a small organization where there are only two administrators, it helps to have some sort of revision control system in place, even if it's only a central log that is checked before updates are made. I can only imagine how complex it must get in a large enterprise with a few or even a dozen programmers working on the same website. Of course an alternative would be to assign different parts of the website to each of us but we just aren't that big.

Monday, October 1, 2007

In which we get slammed

"Slamming" is the illegal practice of changing a person's communications provider without permission, and it can affect customer's local or long distance service. Source: (AT&T). The practice was much more prevalent when long distance was first deregulated but still occurs with alarming frequency today.

We opened a new office in a remote location a year or two ago and set up local and long distance phone service with Qwest. Earlier this year we started noticing charges showing up from long distance carriers that we did not authorize. We went back and forth with the bogus companies who offered small refunds. When advised that we did not want or ask for their services they used intimidating language threatening that we would never be able to use their long distance service again. Duh! Hello! We never wanted their service in the first place. Why would we care?

We resolved this by calling Qwest and having them put a freeze on our account, something we should have done when we first set it up. Telephone service cannot legally be switched from an existing preferred telephone company to a new company unless the new company verifies the switch using one of the following methods: 1) Uses an independent third party to verify an oral authorization to switch. 2) Provides and obtains a signature on a letter that indicates, in writing, that you want to switch preferred telephone companies. 3) Provides a toll-free number that can be called to confirm the order to switch preferred telephone companies. Source (FCC).

So if we desire to file a complaint with the FCC (which we probably won't because it's such a small amount), the bogus carriers must prove that they did not slam us or be forced to pay us a fine equal to 50% of the amounts in the complaint. It is a common slamming practice to send a small ‘refund’ check of a few dollars. When cashed, it authorizes the sender to switch telephone companies. If we cashed such a check then it is our mistake that caused this problem.

Sometimes, sleazy phone companies will trick you into switching carriers by disguising the authorization in a telephone survey. If the person answering the telephone says “yes” to any of the surveyor’s questions, the answers may be taped and used later as verification of authorization to switch preferred telephone companies. Also, someone may have called and offered a free trial offer. The trial is free for 30 days and after that it starts billing every month on your bill.

In our case, the problem was exacerbated by centralized billing and distributed service. The people who pay the bills don't use the same phone service. So somebody at the remote location could have been subject to one of those surveys or a clerk in accounting could have deposited a bogus check without bothering to read the accompanying letter authorizing the switch. It just goes to show you that user education is needed but not always appreciated until the lack of it becomes apparent.