Friday, December 12, 2008

Connecting remote network locations

One of the requirements of an IT Manager working for an organization with remote locations is knowing how to connect the networks from each site. It’s really not a big deal. You just put a VPN between them. It’s very simple as long as you have sufficient bandwidth and know how to secure the connections with a good firewall. Everyone knows how to program firewalls, right?

My first experiencing in connecting two sites was back in the old Novell days. You remember Novell, don’t you? They made one of the first server operating systems designed for PC-based networks called Netware. There are still a lot of long-running Novell servers in small businesses out there even though Novell lost the majority of the market share to Microsoft back in the 90’s.

We used Arcnet back in the day – a coax-based network running at 2.5Mbs with active hubs every 2,000 feet. That’s why we used Arcnet instead of early Ethernet – which was limited to 600 feet end to end. We had multiple warehouses in a small business complex that needed every bit of the distance Arcnet provided. It’s hard to believe that we built that over twenty years ago.

Connecting international sites

Almost every company I’ve worked for since then has had multiple locations, both in their local campus and with locations in distant cities, some international. For the companies that had sites within a metropolitan area we used Frame Relay, an inexpensive way of sharing the public phone network to provide PVC’s – permanent virtual circuits – to each of our offices in the city.

For the international sites, we used 56K dial-up. Yep, you could network two Novell LANs via dial-up for the purpose of exchanging files and email on a demand basis. This was before the days when there was an ISP in every city to provide the connection. The demand became so constant that the long-distance calls from our Mexico plants were sometimes twelve hours a day.

Once Internet Service Providers finally came to the Mexico cities where we had our plants, we dumped the expensive long-distance calls and began setting up point-to-point VPN’s. They were still over the 56K dial-up modems, so they always seemed to be dropping the connection. I am sure it had something to do with the quality of the wiring infrastructure in Nogales and Mexicali.

From dial-up to DSL

When DSL finally came to Mexico, we at last had a halfway reliable method of connecting our two networks. You may wonder why we didn’t do leased 56K lines or T1’s. Remember, this is small business we’re talking about. International leased lines back in the 90’s were thousands of dollars a month. This was also right about the time we were dumping Novell for Microsoft NT.

Connecting remote sites these days is a piece of cake. As long as each location has a high speed connection to the internet, you can share files on servers and send email back and forth all day and night without it costing an arm and a leg. The only real concern is security in connecting your private business locations to the public internet. That’s why you need a good firewall.

We used to use Cisco PIX firewalls but we have switched to Juniper Netscreen’s mainly because they are easier to program and support more features for less money. Cisco to me is like the way IBM was just before they finally got out of the PC Business. They have a huge support structure in place and have to charge more for the same features giving smaller competitors an advantage.

Bandwidth and sharing data

Bandwidth is a critical part of a good VPN connection. It’s not so much the downlink speed as it is the uplink speed. Many people don’t realize that and try to go with a cheap DSL at 768Kbps down and 128Kbps up. Don’t do that. Get the 3Mbs down with at least 512Kbs up. Get more if they offer it. We pay $65 a month for our 3Mbs DSL line as a backup to our symmetrical T1.

Working at the airport is kind of like being on a campus. Although we have fiber between most of our hangars, some are just too far away or across a runway. We couldn’t very well dig up the runway to lay fiber so we opted to use the public network. Connecting a hanger 4,200 feet away is no different than connecting a remote office across the county or on the far side of the world.

As long as both locations have a good Internet connection you can make it look like a server at the other location is in a closet down the hall. This is especially true if you implement DFS – Distributed File System - which caches and replicates local copies of shared files on a Microsoft network. The replication is fault tolerant, fast and reliable even over slow WAN connections.

The VPN makes it happen

DFS is not intended to be used in a collaborative environment where multiple users might have the same file open, making changes at the same time. Just like you would not have two people working on the same spreadsheet on a local network, don’t expect DFS to provide file or record locking capabilities. For that, you need a true shared database application like MS SQL server.

For our new hangar we simply created the VPN between our two firewalls, joined the servers at the remote location to the domain and began the replication process. Our remote employees are able to log in to a local server and have access to shared files at local speeds. We also employ Cached Exchange Mode on their Outlook client to create the local copy of their company email.

The VPN – Virtual Private Network – allows the administrator to perform maintenance on the remote servers and workstations as if they were onsite, because they are inside our network. We use Remote Desktop extensively to provide that support. The sensitive data that flows between our corporate office and our remote locations is secure because of the firewall encryption.

Microsoft technology employed

Where remote employees need to run client-server applications that don’t perform well over WAN distances, we use Microsoft Terminal Services. Our Flight Operations software and our accounting software both use this technology. Employees run their client on a server at the corporate office that is on the local LAN. It uses the same technology as Remote Desktop.

Our new hangar is 110% energy efficient meaning that the electricity it produces from the solar panels is more than sufficient to meet the needs of the electrical systems we have there. We are able to return 10% of the electricity to the city grid. The cameras on the security system are also available to our local authorized network users and are shared for executive home viewing.

Ordinarily I would not mention details like this from our new hangar but the company has gone public with it so if you would like to know more, you can read about it and view it online. We are very proud of the fact that it is the first platinum LEED certified aircraft hangar in the world. My part in the construction was minimal. I just made sure we are well connected and secure.

Tuesday, December 9, 2008

Tech support at the small company

When I first start with a new company I can usually count on my day always being busy. No, hectic is a better word for it. I have always worked for small companies that either did not previously have a tech support guy or that fired the previous support person due to incompetence or an unwillingness to do things the company way. That’s not a good way to treat an employer.

When the employees find out that there is someone on the payroll that knows what they are doing with computers, it’s as if the floodgates open. All the pent-up frustrations of not knowing how to do something with the computer or the network come to the surface and I am deluged with unending calls and requests for meetings to discuss their issues and solve their problems.

The small company mentality

Small companies are interesting to work for. They are quite a different animal from the large corporate environment where there are department and inter-departmental politics. Sure, some of that exists in the small company but for the most part, when you are supporting less than 100 computers, you can count on wearing multiple hats and having to be the expert in all of them.

For example, in addition to the approximately 100 computers including all kinds of laptops, I support a dozen servers in several locations, the LAN at each location, the WAN, the routers, firewalls, switches, fiber connections between hangars at the airport where I work, the email system, the SQL server database, the website, the intranet and all the phone and FAX systems.

Need more than tech skills

When you are the sole tech support guy, there are a multitude of soft skills that are needed to survive in the small business environment. Number one on the list is the ability to relate well to people. Most people don’t care about technology. Computers do not fascinate them. To them, the stuff we work on is nothing more than a tool. They could care less about how it really works.

Your success in helping a frustrated co-worker deal with some problem on their computer is in direct relation to how well you can deflect abuse or blame. I can’t tell you how many times I have heard or have had implied, “It’s your fault that it’s not working.” It’s a real talent to be able to accept that responsibility without taking offense. “No problem – we’ll get it working soon.”

Dealing with executives

The favorite part of my job is dealing with the upper level CEOs and Vice Presidents. For the most part, they are completely helpless when it comes to technology and they hate that feeling of being powerless. If you want to experience some real stress, try taking the call from the CEO when he can’t get the video projector working in front of several multi-million dollar clients.

Executives travel and don’t do well with remote connections. Although it has become easier in the last few years with wireless in the hotel rooms, leave it to the travelling VP to always find some way of messing something up with his Outlook client. A simple change in the view from 100% to 200% can cause a major freak-out with claims that they didn’t do anything. Fix that!

Benefits of small company work

In spite of all the stressful aspects of doing tech support at a small company, there are some major advantages that you won’t find in the big companies. Even though I am a Microsoft certified systems engineer, I enjoy the company understanding and support of a regular budget for outside consulting with other engineers when working on major infrastructure changes.

And, almost always, after a year or two, I am able to convince the boss to allow me to train a junior assistant to take over the day-to-day helpdesk issues. It usually ends up being the son of one of the owners or executives, but that works out just fine. I can then focus on network and server support, concentrating on long-range planning for anticipated growth or disaster recovery.

Summary and conclusion

After nearly thirty years of working for small companies providing tech support, I can endorse the career choice completely. Sure, there is a limit to how much you can earn, but there is also a much lower level of expectation and almost always a higher level of appreciation from those who run the company. The small business is usually run with a very family-friendly atmosphere.

Maybe my experiences with tech support in the small business world have been unique or maybe I have just been blessed, but I no longer miss the idea of working for the huge IT department in corporate America, especially with all the economic concern that we live with today. No job is completely secure, but being the only computer guy for a small company is a pretty good gig.

Saturday, June 21, 2008

Real world example of scope creep

Scope creep is defined as the tendency of a project to grow in scale and complexity as more individuals get involved. It also occurs as the details of the project are presented to the project owners who requested it, usually management, who then say, "Can you also make it do this or that?" Let me give you an example that happened to me just the other day.

I am the IT Manager for a private jet charter management company. Several years ago we added Boeing Business Jets to our fleet. These are larger than the Gulfstream aircraft which comprise the majority of our aircraft. A BBJ is a 737 that is tweaked out with tens of millions of dollars worth of custom mods that make it into a flying luxury yacht for the very wealthy.

Of course a bigger aircraft requires a bigger hangar. So we built one. No, it's not a simple project. It requires a lot of environmental approvals and just the right touch with the airport authorities. An older and smaller hangar was purchased and demolished and the new one has been rising in its place over the past year. It will house the BBJ and two smaller G550 aircraft.

Getting the details defined

From day one I offered management my assistance in defining the network and communications requirements. "No thanks", I was told. "The building contractor has that all taken care of." I sensed trouble and kept following up with occasional emails over the past year asking specific questions like how they would like our sites connected and what the phone system would be.

I confess I played CYA with these emails, documenting each offer of assistance with specifics of what would be needed to make it all work - switches, routers, VPNs, PRIs, VoIP phones, wireless access points and a domain controller for local authentication and file replication. I suspect that the verbiage about wiring closets and cross connects just went over their heads.

This week I received a call from one of the subcontractors wanting to know how many network drops were needed and where exactly they would be going. Did I freak? You bet I did, but I managed it in a very professional way. It was obvious that the contractor had failed in planning properly for all the electronics involved in the new building. Has this ever happened to you?

Managing an out of control project

I told the wiring contractor I would get back to him. I fired off an email to the site project manager, an employee of our company, notifying him of the situation. He assured me that they had provided all the necessary details to the general contractor and it was all included in the plans. Somehow copies of plans don't always make it down to subcontractors, do they?

Next the phone contractor calls and asks, "Where is the MPOE?" There is no physical wiring from the phone company in the building yet. "Let me get right back to you on that," I respond. Is it panic time yet? The building is supposed to be occupied in sixty days and they haven't yet arranged for voice and data to the outside world. Oh, and no phone system has been chosen.

That's it. I call for a general meeting with the contractor, the project managers from all sides and the subcontractors. It turns out the project manager from our company simply had no clue about networks and phones. He thought the contractor had it all handled. I shake my head in amazement. How can you build an expensive hangar and not plan for the damn network?

Here's where the scope creep occurs

During the general meeting to resolve the network and phone issues, the various kinds of phone systems are discussed. I notice out of the corner of my eye that the VP who's baby this is begins to look uncomfortable when we get close to finalizing on a stand-alone VoIP PBX. "What's the matter?" I asked. "Can I pick up the phone and call an extension back at the main office?"

Our existing phone system in the main hangar is twelve years old. It does not even support a PRI (T1). It also does not support remote locations. A single building wiring project just turned into a multi-building job. New phone system for both buildings and new wiring in the old to support VoIP. I was looking for a good reason to upgrade. Its funny how things work out.

Now I have to sell it to the CEO. "What! You want to spend $60,000 on a phone system for this building? We only have a few employees in the new hangar. Why do we have to replace the phone system here?" Ah, the joys of being an IT Manager. If only someone had listened to me from the beginning, this could have all been planned for and budgeted. Now it's a shock to all.

Summary and conclusion

You can draw all kinds of conclusions about how poorly this project was managed. I'll point out one right away - poor communications. But, I've got to tell you after thirty years in this business that this is not all unusual. I've just never seen it happen on this large a scale before. CEOs and VPs are busy with their day to day tasks. Delegating everything without follow-up doesn't work.

In addition to poor communications, the details were undefined in advance. Nobody knew what kind of phone system was wanted or needed. Nobody knew or asked how we would connect our two networks. Wireless access was not even considered. The subcontractors are now overjoyed because they get to sell us a whole lot more equipment than they thought when they were hired.

It all works out in the end. It's only money, right? Unfortunately, it's all too typical of how some large successful companies run projects - everyone likes to delegate but some decisions will always need to be made near the top. That's called leadership but it's hard work because it means dealing with uncomfortable details. After all, that's what IT Managers are paid to do, right?

Monday, June 2, 2008

Tech Republic posts for May 08

Blogging at Tech Republic was a little light this past month. I have gotten myself deeply involved in a disaster recovery planning project that is taking a lot of my time and energy. The project will in all likelihood exceed $100,000. The hardware is looking to be about $60K or $70K. I'm looking at several outside companies to provide the DR planning expertise.

We are looking at implementing Virtual Server technology at either the remote site or back in the main office and then grandfathering the old servers to the remote location. We are experiencing scope creep and considering upgrading our Exchange Server to 2007 in the process. This is quickly becoming a very complex project but I'm enjoying managing it.

Num

Date Posted

Title

1

2008-05-09

Fire suppression for the server room

2

2008-05-10

Setting up a remote hot site

3

2008-05-30

I only read the stories for the comments

4

2008-05-30

New user guide to TechRepublic


I'll confess here that perhaps the real reason blogging on tech Republic has been light is because of the "attack and castigate" mentality of some people who read and comment on blogs. I wrote about it on post number three on the list above. It seems to be so prevalent on many forums and blogs today. It's as if a reader feels that they must challenge whatever the writer presented.

It takes all the fun out of blogging. It has made me seriously think about bringing my blogs back from Tech Republic to my own blog. Here I can write in a bit more relaxed manner, simply sharing some of the things I learn and discover about disaster recovery or any other project I am working on. If it's not interesting, you don't have to read it, but it helps me to write about it.

Saturday, March 8, 2008

Memories of an old tech guy

I have been posting all my tech articles on Tech Republic these days. But I put so much work into this one that I wanted to enhance it with photos here on my own blog. This post was entitled, "Old tech guys are slowly fading away" on my Tech of all Trades blog.

I love visiting computer history sites and watching computer history shows. Why? When I visit these sites I gain a perspective on a part of my life that I did not have at the time I was passing through it. For example, when I first started programming in Applesoft on an Apple II computer, I thought it would be a great business language. Ha!

I know I am going to be dating myself when I bring this up, but humor an old tech guy for a few minutes . I remember when we used to sell software on cassette tapes. We put it in baggies and hung it on a peg board on the wall, right next to the the Commodore PET and the Apple II.

People would bring in the TRS-80 computers (we called them trash-80) and ask us to repair them because the local Radio Shack didn't service their own stuff
. What year would you say this was? If you guessed 1978 you would be right. I'll bet that's before some of you were born.

When I got out of college I went to work as a programmer for some "old school" programming shops. I wrote in RPG II on an IBM System 3 and in COBOL on a Sperry Univac 90/30. That machine was a dinosaur even then. The hot new technology was writing in BASIC and Datashare on a DataPoint ARC network.

I used to love to visit COMDEX in those early years of the microcomputer. I was amazed to see all the technology that was coming out. With thousands of other geeks we ogled over the new Micromodem II for the Apple, which sold for about $300. Since I worked for an early computer store I got it for a lot less. My whole world changed.

I can't tell you how many hours I wasted "surfing the net" back in the late 70's and early 80's. No, the Web did not yet exist and most people had not heard of the Internet. We dialed up places like CompuServe and "The Source" or would just connect to TymNet or TelEnet to see whose network we could log on to. Was that hacking? Maybe.

Rather than bore you with my old tech guy memories, perhaps a short list of some of my favorite computer history sites might be helpful if you are interested in learning more about the history of the personal computer. But be careful, if you have any work to get done today, this trip down memory lane will seriously sidetrack you.

1. The Computer History Museum- The online home of the museum on Shoreline drive in Mountain View. Go to the Exhibits section. You can spend hours viewing the collection of marketing brochures. I especially love their time lines. They even have a great YouTube channel. Careful - I warned you this could be detracting from real work.

2. Apple Computers -Because a big part of my early career involved Apple II computers, I like to include Steven Weyhrich's site on Apple II History. You can find other sites like Apple-History.com, but it hasn't changed much lately. The Apple Museum is a better site and the Wikipedia article is great. Where is the "official" Apple history site?

3. Old-Computers.com - One of my favorites. There are nearly a thousand computers in their museum. Use the index on the left-hand sidebar. The articles in the history section are great, the forums are active and something unique that I haven't seen anywhere else - a major list of collectors from all over. Great if you have an old computer to buy or sell.

4. Computer Science Lab- John Kopplin put together a four part pictorial of computer history through the early 80's. Some of the photos are rare which I have not seen elsewhere. The accompanying descriptions could be taken from a college lecture on the history of computers. The lecture ends as the PC was getting started but is well worth a visit.

5. Computer Chronicles - Who can forget this great TV series from Stewart Cheifet? It aired from 1981 to 2002. Well, you may have never heard of it. You can watch many of the episodes online at the Internet archive. I highly recommend the episode Apple II forever, one of my favorites. More on the history of the show is on stquantum.

6. Old Computer Museum - Although you can find this site from the Old Computers club (#3 above), it is worth mentioning as one of the best organized and presented. This collection of Boris Serebrennikov is outstanding. If you have an old Lisa or even an Amiga (still a popular retro machine) he is interested in hearing from you.

7. The Computer Collector - This is a fairly complete list, useful to those who have old machines to buy, sell or trade. It is also an enormously wealthy site for computer history buffs. Many of those who buy and sell computers have great historical information about the computers they worked on. Lots of great personal history stories can be found here.

8. IBM PC Official History - It still amazes me how many people believe that the IBM PC was the first microcomputer. We used to laugh at those who thought our industry was "legitimized" when IBM finally made their entry in 1981, easily five years after Altair, IMSAI, Cromemco, Apple, Commodore, Radio Shack, Atari, Altos and Vector Graphic.

9. Personal Computer in TV commercials - The download squad has put together a collection of TV commercials for personal computers, some of them going back to the early 1980's. What a hoot! The early William Shatner piece has been removed but the original 1984 superbowl ad introducing the Macintosh is there as well as many others. Enjoy!

10. Old Computers.net- This list could go on and on - and it does if you Google it - but this one needs to be included in my top ten list of sites to visit. An extremely popular site, it is billed as the "Obsolete Technology Website" it includes great links not found on any of the above sites including the Intel museum. Thank you Steven Stengel.

I know I've missed your favorites. Add them in the comments. Also, be sure to check out the resources for "Dinosaur Sightings" on Tech Republic.