I’ve been interested in Symform’s concept of cooperative storage since I first saw it scribbled on a napkin and a couple of powerpoint slides. When I saw it I knew immediately that it was the Eureka! moment that business continuity had been waiting for. I got in on a very early beta, experienced some early beta type snafu’s and then the product released and I started some of my clients on it. It worked and cooperative storage was born.

Of course it was a 1.0 product and so the feature set was pretty basic and one of the glaring things that was needed was bandwidth control because if your data set was large or your Internet use heavy people felt the tug as they wrestled for bandwidth. Now the product has expanded a couple of times and we have full bandwidth control over incoming and outgoing data flows.

Unfortunately for me, this brought forth the glaring reality that we didn’t understand the relationship between contribution, synchronization, cloud folders and how they impact one another based upon our choices with the bandwidth sliders and the backup software we were using. Symform gave us all rope we asked for and hang ourselves we did. How did we know we hung ourselves?

If the Symform Synchronization Service doesn’t stay started, you have problem.

If you have a lot of pending files, you have a problem

We had a problem. We weren’t sure exactly what the problem was, so we called up the guys at Symform and got ourselves a lesson in bandwidth management, Symform style. So here’s what we now know:

• Synchronization uses upload bandwidth
• Contribution uses download bandwidth
• Contribution also uses upload bandwidth
• Synchronization service requires a certain amount of bandwidth be present to function at all
• Dedicating small amounts of bandwidth during the day will not help things
• Symform application has an awesome log file to help you figure out your bandwidth requirements
• Storagecraft, our backup software of choice changes the blocks, so Symform can’t always de-duplicate our files.

Let’s start with the log and work backwards. The log file you want is found at C:\Program Files (x86)\Symform\Node Service and is called symformsync.log. If the Symform Synchronization Service is not running then this log will show why it stopped. There will be a bandwidth requirement listed there for successful upload. Wish I could show you one but ours are all working now and the logs have over written. But look at the bottom of your log file, if you’re in this situation and you’ll see what I mean. You will need to set your minimum bandwidth to this number, even during business hours.

Now the thing that had us confused was not taking in account that the contribution service used both upload and download bandwidth. Our simple brains said – incoming traffic. Completely forgetting that if those files turn out to be needed by someone they will need upload bandwidth. But the other thing that caught us was that Storagecraft moves blocks, because when that happens Symform’s de-duplication technology sees them as new files and so the whole thing gets uploaded. We had really hoped that our full backup could be deduplicated so that after the first one, it wouldn’t take so long but that is not going to be the case. This changes things dramatically.

Here’s a sample of what our backup looks like. You get the idea that it’s a lot and it sometimes has some pretty big changes.

A quick search on the Internet comes up with a lot of references for upload calculators. That first full image looks like this in the calculator. Good thing we have more bandwidth than that.

We’re allowing 800K for 9 hours a day (business hours) and 1.8MB for 15 hours a day. This is a story problem…It’s about 21 hours at 800K or 9.6 hours at 1.8MB. In reality its going to be somewhere in between because we have contribution taking some of that bandwidth. It’s going to be a while for that full backup to get to the cloud. But not so long that we can’t get one up there over a weekend. Which means we can do a full weekly backup. In an average week day we have a total of 5GB of hourly incremental data to upload. Which is about 3 hours at 1.8MB and bit over 6 hours at the slower speed. That is probably stretching the bandwidth about as far as it can go, so we may find that we need to reduce those incrementals to every 2 hours instead. Time will tell.

Now one final point of knowledge. Understanding the Symform Cloud folder in relation to your Backup folder. Say you’ve goofed up as we did, you’ve got a lot of pending files waiting to upload and you realize that you need to make some changes. So you setup a new backup strategy, change the bandwidth sliders and the new files start populating into the folder. The files that never uploaded are so old by now anyway that you clean out the backup folder for a fresh start at this. What happens now? Not a thing. Symform is confused, looking for those files that you removed that were in a pending state and so it waits for their return and none of your new files get uploaded. 

Here’s where we learned the relationship between the cloud folder and our local backup folder. The cloud folder is a list of files that Symform is going to pull into the cloud and their status. The local backup folder contains the files that you want Symform to upload. The distinction is important.

To get Symform to let go of the idea of backing up the pending files that no longer exist, you need to delete your cloud folder. Sounds scary, doesn’t it? Then create a new cloud folder and point it at your local backup folder for a fresh start. It’s at this moment, just when you thnk that you’ve deleted files in the cloud and will have to wait for the files that already backed up successfully to upload again, that the magic of Symform kicks in. The new cloud folder populates itself with the fresh list of files to be uploaded. The Symform control service now takes a look at the files and <insert angelic LAAAA> discovers that it already has these files uploaded and marks
them as such, saving you hours and hours of upload time. How? Because of two nifty features, de-duplication (do I have this block – yes I do) and recycle bin. In the recycle bin, (at the control center) they will hold your files (or more accurately the location of your files) for 15 days before giving the order to destroy them. So provided you create your new cloud folder within that time frame, you’re golden. Your files that had uploaded previously will all quickly reset to uploaded status.

Get the sliders and your backup schedule set just right in relation to the bandwidth you have access to and Symform will perform like a champ giving your clients business continuity options they couldn’t afford before.

—–

Need more help with this or some other sticky situation? Click to hire us.
We’re Third Tier. We provide advanced Third Tier support for IT Professionals.
Get Support Blog Twitter Facebook LinkedIN