It's been a while ago and some good things have happened in the meantime, but I still think Nick Howitt's suggestion for moving a mail system is an excellent one.

For e-mails, if you are going down the cyrus-imapd route in 6.3, there are a couple of ways of doing it. The fast way is to tar up /var/spool/imap and /var/lib/imap, then untar them into 6.3 and use the reconstruct command to rebuild the mail boxes. It is per-user but does not need passwords and it is much quicker than the other option.

This is a fast route without any troubles. It uses cyrus-impad which is an enterprise level tool so it should be safe. Drawbacks - as Nick mentioned - are that you will re-import the entire history with all its flaws. Also, all mails will probably be categorized 'unread' after re-importing. But it's a clean and very fast process. I moved a 10GB mail system with hundreds of subfolders in just 30 mins.

Nick has described the process well but I will elaborate a bit here.

First, download cyrus-imapd if you don't have it.

yum install cyrus-imapd

Then, on the ClearOS 5 system, tar up the mail files

tar -cf mailtemp /var/spool/imap /var/lib/imap

Move the file mailtemp over to your ClearOS 6 box by using FTP or WinSCP, or use a USB stick. Install the file on the ClearOS 6 side in /.
On the ClearOS 6 box, in order to untar, type

cd /

tar -xf mailtemp

Your files will now be present, but it isn't a mail system yet. You will do a 'reconstruct' in order to achieve that

cd /usr/lib/cyrus-imapd

./reconstruct

Then just lean back and relax while you see your mail system being re-constructed.
See also: http://www.techrepublic.com/blog/data-center/reconstruct-corrupted-mailboxes/

Hope this helps to overcome any possible fear of moving your mail system to ClearOS 6.

Unfortunately, it's still only a tool that we use internally to support customers. We just don't have enough data points to put it out there as an end-user tool ... little gotchas here and there. For the wider audience out there, the ClearCenter support team can take a ClearOS 5.2 export and transform it into a ClearOS 6 import (users, groups, passwords, and Samba). It's still a semi-manual process, hence the need for support.

Any hope for import LDAP-Data from 5.2 to 6.5?

I have found this in the tracker: http://tracker.clearfoundation.com/view.php?id=1201
Its closed for the upcoming 6.5 Beta1

Greetings Alex

Rolfb, sorry if you have been waiting a long time for this. It is not my intention to mislead...I simply restate what information I see from the devs here in the forums.

Alex, ClearOS has always been CentOS / Redhat based...
http://www.clearfoundation.com/Software/distribution-timeline.html

Ruud, the app framework in 6.x has been completely rewritten, whilst it is possible to transfer the configuration files across, there is the possibility they are out of date for the new package versions you'll be running, or whether the apps will interpret them correctly.

FYI all the ClearOS configuration is kept at /etc/clearos/ in 6.x. Here you can find the firewall settings / flexshare / samba / proftpd etc. But things like your interface names and network configuration have changed and now handled by the network manager, and a new file at /etc/clearos/network.conf

Data is still stored at /var/flexshares/shares and /home, any migration plan should consist of backing this up...clean install, recreate network config and roles, and reimport data.

You'll also want to use mysqldump if you have databases, and reimport those... You'll also want any virtual host configurations from /etc/httpd/conf.d (same in 6.3)

Hi,

I have also 'caught' myself looking for other solutions as these probably have the same impact as doing a fresh install.
On the other hand, experience learns that a system (5.2) that has been around for suh a long time and has 'grown' with tweaks etc. it is better to do a fresh install and start al over again. Maybe (for sure ) some of the needed tweaks on 5.2 will be obsolete or functional out-of-the-box for a version 6.x install.

Perhaps it is an idea to create a howto per module / functionality on how to upgrade / transfer over to the new installation. e.g. how to transfer the firewall rules, eth interfaces, mail (there is already one), mysql, etc.

regards,
Ruud.

The devs have already said they hit a show stopper which prevents a yum upgrade.

Tim & team,

if this just is not going to happen, then please just say so. I've given up hope and am very disappointed.
So, since I just have no option but a fresh install, I might as well look around for another distro.
Not being able to upgrade is maybe not Clears fault, but keeping people waiting for months (or is it over a year already) is unforgivable.

Hi

I too face a possible upgrade, but not in a hurry.

If this cannot be scripted or otherwise automated due to the significant changes, perhaps a detailed "how to" with checklists could be written and a few select tools/apps made available.

I used imapsync when I migrated from our old OSX 10.3 server, which had a fairly clean Postfix/Cyrus IMAP setup. The drawback, as noted earlier, is that you need every user's password. There is IIRC a method to have an cyrus admin or 'superuser' to at least pull all the messages, but I could not get that to work.

Also, despite claims I have seen in many places, I have never been able to get the cyrus reconstruct utility to preserve flags.

A tool that would allow the accounts/passwords/group memberships to be transferred would be great. Even if this requires seperate export from 5.2 and import to 6.x steps

Files in flexshares could be moved using some form of sync (ie rsync)

TIA
Ken

Tim Burgess wrote:

Just to highlight that upstream RHEL do not offer upgrades between major versions (5>6) because the underlying changes are just too great.

Oh, thanks tim, that was new for me but you are right. But then its lookin like it was a bad decision to change to (CentOS/Redhat) some year ago as it was called Clarkconnect.
I dont know such Problems from debian or ubuntu. Maybe this is a big advance for zentyal, collax and the other debian-based "Small Business Server".

Greetings Alex

I'm not sure how a fresh install would work for our office. We're using a clearbox (the hardware) and were told there'd be an upgrade tool. So I'm wondering about an eta since the only way to update is via flash.

Hi Tim,

is this a way to say: "hey, you can wait for the script, whenever it'll come. But it won't work anyway, so just go for a fresh install?"

Thanks Tim. It just seems like there should be some type of way to accomplish this. Especially for someone like me looking to replace hardware and all. I can handle getting the data moved over, but with ldap there's got to be some type of procedure, however complex, that we can follow to get the user accounts to the new machine.

Just to highlight that upstream RHEL do not offer upgrades between major versions (5>6) because the underlying changes are just too great. Peter was looking at a migration tool for users but even still there are also a lot of changes underneath ClearOS in version 6. LDAP plugin architecture, group policies, preparation for Samba 4, etc that makes it difficult to just 'upgrade'. Even if a script was available it can be impossible without breaking something...

I think its ok to use imapsync if you migrate to zarafa. Thats alway the normal way to go to zarafa.
But the migration from the Infos stored in the ldap like username,password and the infos from the domain-membercomputers is the important thing for a new OS-Version.
I dont talk from a nice migrationapp but any documentation if its possible, when its possible and a howto if its possible.

ClearOS was released 7 Month ago! What is with possible new ClearOS-users if they read this and think "maybe the same Problems with migration to ClearOS7?"

I like ClearOS very much, but this is a big disappoint.

Greetings Alex

Thank you... imapsync sounds like a good tool and I'm sure I'll have a use for it in the future, but for now I've still got to hold off until they come up with a utility, script or something to help migrate the users. I want the new version, but I don't see enough benefits to go through all this. If I was starting from scratch, it would be GREAT! But I don't know that it's worth the headaches just to say I've got the latest and greatest when everything is working fine.

My set up is far more simple than yours. I'm just a home user with simple filesharing and a couple of remote family users. That side of the change was not a big issue. The biggest issues were the lack of full mail integration and sorting out the problems resulting from ClearOS doing a standalone installation rather than a gateway installation.

imapsync is a great little utility which can synchronise imap mailboxes on different machines. It can bu used for various purposes like backing up mailboxes/mail servers or having mailservers in different locations in sync so if you have a user travelling between different sites of yours, his e-mails always appear to be local to the site.

For e-mails, if you are going down the cyrus-imapd route in 6.3, there are a couple of ways of doing it. The fast way is to tar up /var/spool/imap and /var/lib/imap, then untar them into 6.3 and use the reconstruct command to rebuild the mail boxes. It is per-user but does not need passwords and it is much quicker than the other option. The other option I used was to create a ClearOS VM on a local PC (I had a play one for testing 6.3). Set up all your users on the VM then use the imapsync command to synchronise the live mail with your VM. This can be largely done in advance then you do a final imapsync (it can just do "changes") just before you bring your 5.x machine down (after stopping fetchmail and, possibly, postfix). After you have loaded 6.3 you can imapsync the mail onto the 6.3. This is a "cleaner" solution as it will not bring across any dross but it is slower. If you have the luxury of a spare machine you can load 6.3 and imapsync the 5.2 mail to it while it is running doing a final imapsync just before you shut off the 5.2 machine. It should then be very quick to bring the 6.3 machine on line. The problem with imapsync is that you need to know all the passwords and it is relatively slow.

Nick Howitt wrote:

Wiping the whole server is a bit drastic. I tarred up /var, /usr, /etc and /homes (very little is on /homes and I do not use flexshares under /var). I imapsync'd all the e-mail onto ClearOS running in a VM on my desktop and did the same in reverse to get the mail into 6.3. Then I did a fresh installation of 6.3. I hit a real nasty as I installed onto my gateway which was still connected to the internet. I had disconnected the LAN cables as I wanted to copy over /etc/ethers form my old installation before any LAN device tried to get a lease. In its wisdom, ClearOS did a standalone installation with no firewall without asking me while directly connected to the internet. Very naughty and it has left a few problems which I discover every now and then.

You also need to consider the features you want from 6.x as some are still missing compared to 5.2.

I played around with this for a while before I just gave up and said I'd wait for a script or something that could shed some light.

Your process sounds a bit like a pain, but if it works I don't mind killing a few hours to knock this out.

That being said, when I was looking at doing this, I was just going to put it on new hardware. But I was never able to get any ldap sync stuff to work. I've got a relatively small install with only about 25 users, but their allover the world, not in one central location and I'd hate to go through the tasks of having to recreate accounts, reset passwords and then deal with the group memberships and permissions problems that I'd have (not to mention the late night calls that come with doing IT support internationally). I don't quite understand what you meant by "imapsync'd"... Can you elaborate on that a little bit? Would that do anything to help with users and groups?

Thanks!

I hate saying so, but I'm still waiting on the migration to 6.x from 5.2 too. Feels like a dark void of information in regards to this topic, where most of the updates are found in forum posts and not any kind of official front page release.

Is there a tentative ETA on a completed migration path?

Wiping the whole server is a bit drastic. I tarred up /var, /usr, /etc and /homes (very little is on /homes and I do not use flexshares under /var). I imapsync'd all the e-mail onto ClearOS running in a VM on my desktop and did the same in reverse to get the mail into 6.3. Then I did a fresh installation of 6.3. I hit a real nasty as I installed onto my gateway which was still connected to the internet. I had disconnected the LAN cables as I wanted to copy over /etc/ethers form my old installation before any LAN device tried to get a lease. In its wisdom, ClearOS did a standalone installation with no firewall without asking me while directly connected to the internet. Very naughty and it has left a few problems which I discover every now and then.

You also need to consider the features you want from 6.x as some are still missing compared to 5.2.

Hi,

any news on this subject?
I'm about to give up, and look around for other solutions.
We're still hooked to 5.x, not able to upgrade to 6.x (how far is development).
I might as well just wipe the whole server and go for a fresh install ;-((((

:angry:

Really looking forward to this. Am still hooked onto to 5.2 across all my installations because of this.

It has bubbled to the top of the priority list and I really want to have something solid in place for the 6.4 release (November/December).

Hi all,

Any news about the migration tool?

But my biggest upgradeproblem is hot so move a samba pdc-Domain to ClearOS6.3. Its possible to migrate 50 users by hand but it tooks very long time to bring 50 windows-pcs into a new Domain.

I tried many things with slapcat, modifying the ldif-Files and import it with slapadd. I was learning a lot about ldap,samba and the changes between 5.2/6.3 last days.
But finaly I always failed. Does anybody knows the trick?

Many thanks
Alex

Sorry, no ETA yet. Unfortunately, an LDIF backup won't quite work since the access control framework has changed.

Peter, do we have an ETA on this now that 6.3 is also out?

I tried to restore with a standard backup from 5.2 but it failed with a "invalid format" error.

Most of the settings can be ported manually, but I am most concerned about the user accounts keeping their ids and permissions consistent. Would a ldif backup and restore do the magic?

Thanks Tim. As always, you're a wealth of information. I use bandwidth rules and mail retrieval so I guess I'll be waiting before I make the change to COS6. I know there was some discussion a while back about IPv6 support. Do you know if that is either in COS6 or planned for the foreseeable future? (I don't see it in the roadmap section of the project tracker).

To an extent depends what your production environment is??

6.3.0 final release is due very shortly, and I would consider that all major known issues have been squashed
http://tracker.clearfoundation.com

There are still more features to be added perhaps to bring the interface up to the same level of configuration as 5.2...
- advanced bandwidth rules
- web proxy bypass
- mail retrieval

Is 6.2 or 6.3 ready for a production environment?

Julian S wrote:

Does the 6.3 beta include the import logic for a 5.2 config backup? If not is there a new estimate on when it might be available?

No ETA at this time, sorry.

Does the 6.3 beta include the import logic for a 5.2 config backup? If not is there a new estimate on when it might be available?
For me personally I'm most interested in being able to keep the users and (VPN) certificates if possible.

Hi,

Whens the migration tool expected?

Thanks a lot for the update! Once the migration tool is complete, I'll definitely be looking to upgrade my current 5.2 systems.

We looked at the upgrade possibility and it's a bit too ugly.

- Upstream does not officially support upgrades. That's a big warning signal that is hard to ignore.

- ClearOS 6 is strict about using group-based policies. That means unattented upgrades for the Content Filter, Administrators, and Flexshares (which allows user-based shares) is not possible. An administrator has to revisit these apps.

- Samba changes were significant enough to cause issues.

There are other little things as well, but those are the show stoppers. So, we started down the road of implementing import support for the "Configuration Backup" tool. Most of the LDAP and Samba work is already done (and in SVN). There's probably about a week of developing/testing time before the migration tool can be put to a full test. When it's done, you will be able to take a ClearOS 5.2 configuration backup and import it into ClearOS 6.

At the end of the day, it's a work in progress.

Any news on a upgrade possibility yet?

I was also surprised by release 6.2, running 5.2, and an automatic yum checkupdate script daily.
How (apart from checking the website) is one informed about a major update? Not with yum apparently...

Cool, but what about things like exporting user accounts or security certificates? I use a ClearOS machine as a gateway and file server with OpenVPN. I also have it receiving and scanning emails and then forwarding them to another mail server.

I'd like to switch it out for a new 6.2 machine, but I'm not sure how to handle getting the user accounts to the new machine. I hoped to be able to do it using LDAP replication, but the instructions at http://www.clearfoundation.com/docs/developer/architecture/central_management/openldap_replication_settings require kolab which is not a part of 6.2. (I'm very new to LDAP and that how-to was as far as I was able to get on my own). If I can figure it out, I'd love to write a how-to guide for others like myself who can't figure out how to get the user accounts to the new machine.

Thanks again for any help you can provide.

V

Pete promised one around the time 6.2.1 is released to address those migration issues, but i've no idea if that's still realistic

Is there a simple procedure for migrating user accounts and settings from a 5.2 machine to a new 6.2 install? Something like the backup/restore?

Also, is there any timeline for developing an upgrade?

An upgrade is not yet available but is planned for the near future. If you want 6.2 now it has to be a re-install, but note that some features, especially mail, are not yet in 6.2.