Category Archives: Microsoft Exchange

Search and Delete certain Items/Folders from a Mailbox

The Story

During a project of Hybrid migration from Exchange on-premises to Exchange online, I was almost about to finalize the project by moving the last remaining users mailboxes however had an interesting issue to deal with where a user was failing with the following error:

Error:

Error: MigrationPermanentException: Mailbox dumpster size 50.87 GB (54,620,074,576 bytes) exceeds target quota 30 GB –> Mailbox dumpster size 50.87 GB exceeds target quota.

image

After some research it turned out that you can clean the dumpster using search-mailbox PowerShell cmdlet, Sync the user’s object with ADConnect and then continue the migration from the last failure.

To solve the issue, Go on your Exchange on-premises and launch Exchange Management shell

Solution:

First, Let’s see the user’s dumpster and recoverable items

Get-MailboxFolderStatistics -Identity “User” -FolderScope RecoverableItems | Format-Table Name,FolderPath,ItemsInFolder,FolderAndSubfolderSize

image

To Delete the dumpster only use this

Delete dumpster only

Search-mailbox -identity User -SearchDumpsterOnly –DeleteContent

To delete a certain email with certain subject in the dumpster use the following:

Get-mailbox “user”| search-mailbox –searchquery “Subject:’*'” –DeleteContent –SearchDumpsterOnly

image

The cmdlet will search and delete

clip_image001

image

Reference:

https://docs.microsoft.com/en-us/microsoft-365/compliance/search-for-and-delete-messagesadmin-help

MRSPROXY.SVC’FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT. THE REMOTE SERVER RETURNED AN ERROR: (404) NOT FOUND

Symptoms

MRSPROXY.SVC’FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT. THE REMOTE SERVER RETURNED AN ERROR: (404) NOT FOUND

You get an error when you’re trying to setup Hybrid configuration between your Exchange On-premises and Exchange Online.

After I had one issue like this I did some research and used Fiddler/Wireshark to check for traffic I noticed that Exchange was not encrypting traffic and testing the Migration Server Availability was reporting that the MRS service was not listening on the supposed port which is 443.

image

In this case the solution was pretty easy, but still you’ll have to make a hard choice of choosing to place Exchange behind a load balancing with SSL Offloading on or not.

In my case I had to turn off the SSL Offloading on the Load balancer and that alone was enough to get this working.

Resolution:

Make sure that SSL Offloading is disabled on OWA/OA and Load balancer if there’s one.

Other resolutions:

https://support.microsoft.com/en-us/kb/3065754

Installing and configuring IIS SMTP for Office 365 with User License

Prerequisites:

1- Office 365 Licensed User.

2- Windows Server 2016 OS.

3- Public Static IP (To add to the SPF policy to avoid getting emails flagged as spam)

4- Allow outbound port 587 on your Firewall.



Regardless of Windows Operating system you’re going to install the SMTP Role the steps are the same. You’re going to install the SMTP Server feature from the Server Manager as following:


First Launch Server Manager (Windows 2016) and click on Manage\Add Roles and Features


image

Next

image

Click next to choose the installation type, Make sure you choose the Role-based or Feature type of installation

image

Next select the server where you want to install the SMTP Feature

image

Click next to head to the next step, The SMTP feature is normally located in the Features page not the role so proceed further by clicking Next

image

Scroll down in the Feature page until you get to SMTP Server

image

Click on the checkbox next to SMTP Server and add the features which will automatically pop up

image

Click Next

image

Click Next again, Leave unchanged here

image

Click Install and you will get your SMTP Server ready


image

Open IIS 6.0 Manager to Manage your Relay

image

To setup SMTP server for relay through Office 365 follow those steps:


Set up SMTP

Open Server Manager, select Tools, and then select Internet Information Services (IIS) 6.0.

  1. – Expand the current server, right-click the SMTP Virtual Server, and then select Properties.
  2. – On the General tab, select Advanced > Add.
  3. – In the IP Address box, specify the address of the server that’s hosting the SMTP server.
  4. – In the Port box, enter 587 and select OK.
  5. – On the Access tab, do the following:


clip_image001

On the Access tab, do the following:

  1. Select Authentication and make sure that Anonymous Access is selected.

clip_image001[4]

Select Connection > Only the List Below, and then specify the IP addresses of the devices that will be connecting to the SMTP server, such as printers.

clip_image002

clip_image003

clip_image004

Select Relay > Only the List Below, and then specify the IP address of the devices relaying through this SMTP server

clip_image005

  1. On the Delivery tab, select Outbound Security, and then do the following:
    1. Select Basic Authentication.
    2. Enter the credentials of the Office 365 user who you want to use to relay SMTP mail.
    3. Select TLS Encryption.
    4. Select Outbound Connections, and in the TCP Port box, enter 587 and select OK.
    5. Select Advanced and specify SMTP.office365.com as the Smart Host.

image

clip_image007

clip_image008

  1. Restart the IIS service and the SMTP service.

clip_image009

https://technet.microsoft.com/en-us/library/dn592151v=exchg.150).aspx>


clip_image010

Let’s make a test by creating a notepad file with the following Test

image

image

Ref:

https://technet.microsoft.com/en-us/library/dn592151(v=exchg.150).aspx

Russian cyberspies are using one hell of a clever Microsoft Exchange backdoor

Turla APT found exploiting LightNeuron backdoor, a first of its kind targeting Microsoft Exchange email servers.

image

A Russian cyber-espionage group has developed and has been using one of the most complex backdoors ever spotted on an email server, according to new research published today by cyber-security firm ESET.

The backdoor, named LightNeuron, was specifically designed for Microsoft Exchange email servers and works as a mail transfer agent (MTA) –an approach that no other backdoor has ever taken.

“To our knowledge, this is the first malware specifically targeting Microsoft Exchange,” ESET Malware Researcher Matthieu Faou told ZDNet via email.

“Turla targeted email servers in the past using a malware called Neuron (a.k.a DarkNeuron) but it was not specifically designed to interact with Microsoft Exchange.

“Some other APTs use traditional backdoors to monitor mail servers’ activity. However, LightNeuron is the first one to be directly integrated into the working flow of Microsoft Exchange,” Faou told us.

Because of the deep level the backdoor works, LightNeuron allows hackers to have full control over everything that passes through an infected email server, having the ability to intercept, redirect, or edit the content of incoming or outgoing emails.

LIGHTNEURON DEVELOPED BY TURLA GROUP

This makes LightNeuron one of the most powerful tools of its kind, and a tool fit to be in the arsenal of Turla, one of the world’s most advanced nation-state hacking units.

The Turla APT (advanced persistent threat) is infamous for past operations that seem to be pulled out of Hollywood movies. The group has been known to hijack and use telecommunications satellites to deliver malware to remote areas of the globe, has developed malware that hid its control mechanism inside comments posted on Britney Spears’ Instagram photos, and has hijacked the infrastructure of entire ISPs to redirect users to malware.

In a report released today, ESET says that Turla has been using LightNeuron for almost five years, since 2014, which again shows the tool’s advanced capabilities, being able to avoid detection for so many years.

To be fair, the first mention of LightNeuron was in a Kaspersky Lab report on the APT Trends of Q2 2018. However, Kaspersky only described the tool in brief. The ESET report released today shines more light on the tool’s unique capabilities that make it stand out from all other backdoors deployed on email servers up until now.

Researchers warn that LightNeuron is currently being used in live attacks and that Turla also appears to have created a UNIX port –which ESET hasn’t been able to find until now.

The Slovak cyber-security firms said it detected three victim organizations infected with Turla’s LightNeuron backdoor. The company did not name the victims, but provided general descriptions:

– Unknown organization in Brazil
– Ministry of Foreign Affairs in Eastern Europe
– Regional diplomatic organization in the Middle East

A CLEVER WAY OF CONTROLLING LIGHTNEURON

According to researchers, the thing that made LightNeuron stand out, besides being the first backdoor for Microsoft Exchange servers, was its command-and-control mechanism.

Once a Microsoft Exchange server is infected and modified with the LightNeuron backdoor, hackers never connect to it directly. Instead, they send emails with PDF or JPG attachments.

Using the technique of steganography, Turla hackers hide commands inside PDF and JPG images, which the backdoor reads and then executes.

Per ESET, LightNeuron is capable of reading and modifying any email going through the Exchange server, composing and sending new emails, and blocking a user from receiving certain emails.

Furthermore, victim organizations will have a hard time detecting any interactions between Turla operators and their backdoor, mainly because the commands are hidden inside PDF/JPG code and the incoming emails could be disguised as banal spam.

In addition, if anyone had any doubts LightNeuron was the work of Russian hackers, ESET researchers said that in the cases they investigated they found that Turla operators only sent commands to backdoored servers during a typical 9-to-5 workday in the UTC+3 (Moscow) timezone, and took a break from all operations between December 28, 2018, and January 14, the typical Christmas and New Year holidays for Eastern Orthodox Christians –Russia’s main religion.

LightNeuron working hours

Image: ESET

Because LightNeuron works at the deepest levels of a Microsoft Exchange server, removing this backdoor is quite problematic.

ESET released a white paper today with detailed removal instructions.

Useful Powershell script to resolve the X500 address

In migration, Powershell can be a very crucial tool to achieve success and finalize projects within deadline or even fix issues.

During the time of working with Exchange we had lots of issues with users not able to send an email to their migrated colleagues due to some issues with contacts which was caused by the Legacy Exchange DN not being migrated with the user or lost due to some wrong deletion.

Once users try to send an email to that particular user with the missing Legacy Exchange DN. The receiving Exchange server will result an error and send it to the user as NDR message explaining to them that the error is due to not finding the particular address.

image

The solution to this particular problem is very simple especially if it’s couple of users however to resolve the address you’ll need to google and understand the language that Exchange server users to match the original used address in the missing user’s attributes.

The below script would work accordingly with whatever situation that faced me and it became very handy to me.

How to use:

1- Copy the script to a notepad and save as convert.ps1 on Desktop

2- Run script and try to type in powershell convert-X500 then hit enter.

3- Copy and paste the address you got from the error message above.

image

Once you copy and paste hit enter and you’ll get the final result

image

Note: Make sure you remove the @domain.local in the end

Function Convert-X500{ # Define the Legacy Exchange DN here
Write-Host “”Enter your X500 Address here…”” -ForegroundColor Green -BackgroundColor Black
$X500Source  = read-host

# Converts the various strings to the proper syntax
$X500 = $X500Source.Replace(“_”, “/”)
$X500 = $X500.Replace(“+20″, ” “)
$X500 = $X500.Replace(“IMCEAEX-“, “”)
$X500 = $X500.Replace(“+28”, “(“)
$X500 = $X500.Replace(“+29”, “)”)
$X500 = $X500.Replace(“+2E”, “.”)
$X500 = $X500.Replace(“+5F”, “_”)
$X500 = $X500.Replace(“@YourDC.localHere“, “”)

Write-Host X500:$X500}

Hosted Exchange 2013 Tenant setup step by step

 

Starting with Exchange 2013 Hosted .. this is a brief article about how to setup Tenants correctly step by step

1- Create OU

Create New-ADOrganizationalUnit -Name Test10 -Path ou=test10,OU=ExchHosting,DC=lab,DC=com

clip_image001[6]

2- Set OU’s UPN

set-adforest -identity exch01.lab.com -UPNSuffixes @{add=”test10.com”}

clip_image002[6]

Moving to Exchange Management Shell

3- create accepted domain on Exch

New-AcceptedDomain -Name “test10” -DomainName test10.com -DomainType:authoritative

clip_image003[6]

clip_image004[6]

4- Create address book

New-GlobalAddressList -Name “test10 – GAL” -ConditionalCustomAttribute1 “test10” -IncludedRecipients MailboxUsers -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

clip_image005[6]

Note:

Address books once created is virtually separated (That means you don’t have to do anything to separate Address books unless you want them to see each other)

https://technet.microsoft.com/en-us/library/hh529948(v=exchg.150).aspx

clip_image006[6]

  1. If needed then create All Rooms Address List

New-AddressList -Name “Test10 – All Rooms” -RecipientFilter “(CustomAttribute1 -eq ‘test10’) -and (RecipientDisplayType -eq ‘ConferenceRoomMailbox’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

clip_image007[6]

6- And time to create All Users Address List

New-AddressList -Name “Test10 – All Users” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘User’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

clip_image008[6]

7- The All Contacts Address List

New-AddressList -Name “Test10 – All Contacts” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘Contact’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

clip_image009[6]

8-The All Groups Address List

New-AddressList -Name “Test10 – All Groups” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘Group’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

clip_image010[6]

9-Now for something quite useful: Offline Address Book

New-OfflineAddressBook -Name “Test10” -AddressLists “Test10 – GAL”

clip_image011[6]

  1. Mail address policy

New-EmailAddressPolicy -Name “Test10 – EAP” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com” -IncludedRecipients “AllRecipients” -ConditionalCustomAttribute1 “Test10” -EnabledEmailAddressTemplates “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”,”smtp:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}m@test10.com” -EnabledPrimarySMTPAddressTemplate “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”

clip_image012[6]

Set current enabled mail in the test10 organization’s smtp address to test.com

Set-EmailAddressPolicy -Identity “Test10 – EAP” -EnabledPrimarySMTPAddressTemplate “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”

clip_image013[6]

If you’re connected from the Normal Exchange management shell you’ll normally get an error stating that users don’t exist on your Active Directory which is normal since the session you’re connected to is connected to the top forest users and can’t see the address book that’s isolated from other tenants.

clip_image014[6]

You will need to open Powershell (not Exchange management Shell) and use the following cmdlet

Let’s connect to exchange, but before we have to add one of our Organization test10 users to the Organization Management Group in order to be able to administer his organization

$cred = get-credential

$Session = New-PSSession -Authentication basic -Credential $cred -ConnectionUri https://mr.moh10ly.website/PowerShell/ -ConfigurationName Microsoft.Exchange -AllowRedirection

Import-PSSession $Session

clip_image015[6]

clip_image016[6]

clip_image017[6]

11- Address book policy

Now we need to create the Address Book Policy

New-AddressBookPolicy -Name “Test10” -AddressLists “Test10 – All Users”, “Test10 – All Contacts”, “Test10 – All Groups” -GlobalAddressList “Test10 – GAL” -OfflineAddressBook “Test10” -RoomList “Test10 – All Rooms”

clip_image018[6]

12- Optional

New-Mailbox -Name ‘Test10 Conference Room 1’ -Alias ‘test10_conference’ -OrganizationalUnit ‘ou=test10,OU=ExchHosting,DC=lab,DC=com’ -UserPrincipalName ‘test10conf1@test10.com’ -SamAccountName ‘test10_conference’ -FirstName ‘Conference’ -LastName ‘Room 1’ -AddressBookPolicy ‘Test10’ -Room

clip_image019[6]

To mark this conference room for the tenant test10 I’ll have to use this cmdlet otherwise it won’t show in the address book of test 10

Set-Mailbox test10_conference -CustomAttribute1 ‘Test10’

clip_image020[6]

Set calendar settings for the conference room we have just created as following

Set-CalendarProcessing -Identity test10_conference -AutomateProcessing AutoAccept -DeleteComments $true -AddOrganizerToSubject $true -AllowConflicts $false

clip_image021[6]

13- Create mailboxes

$c = Get-Credential

New-Mailbox -Name ‘Mohammed Test’ -Alias ‘test10_mohammed’ -OrganizationalUnit ‘ou=test10,OU=ExchHosting,DC=lab,DC=com’ -UserPrincipalName ‘mohammed@test10.com’ -SamAccountName ‘test10_mohammed’ -FirstName ‘Mohammed’ -LastName ‘HAmada’ -Password $c.password -ResetPasswordOnNextLogon $false -AddressBookPolicy ‘Test10’

clip_image022[6]

Now once I use this script the user will be created and you’ll be able to see In from the ECP

clip_image023[6]

But you won’t be able to see the user in the address book of the tenant since we didn’t apply the tenant which this user belongs to

clip_image024[6]

To do so we’ll have to use the following CMDLET

Set-Mailbox arthas@test10.com -CustomAttribute1 “Test10”

clip_image025[6]

clip_image026[7]

Preparing, Installing and Configuring Exchange 2016 with DAG on Windows 2016


So Exchange 2016 preview version came on MSDN and I decided to give it a try along with the DAG …

Previously in Exchange 2013 I used to have an issue with the fast search on DAG as in some cases it used to stop and cause the original database and copy to report not healthy.

Here I wanted to Install exchange 2016 on new windows edition along with configuring DAG and observe the database’s indexing status.

So to start, I used the available Microsoft Technet related to Exchange 2016.

I’ll launch Powershell as adminsitrator an start by installing the requested software

Install-WindowsFeature RSAT-ADDS

From <https://technet.microsoft.com/en-us/library/bb691354{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}28v=exchg.160{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}29.aspx?f=255&MSPPError=-2147217396>

clip_image001

Windows 2012/2012R2

Install-WindowsFeature AS-HTTP-Activation, Desktop-Experience, NET-Framework-45-Features, RPC-over-HTTP-proxy, RSAT-Clustering, RSAT-Clustering-CmdInterface, RSAT-Clustering-Mgmt, RSAT-Clustering-PowerShell, Web-Mgmt-Console, WAS-Process-Model, Web-Asp-Net45, Web-Basic-Auth, Web-Client-Auth, Web-Digest-Auth, Web-Dir-Browsing, Web-Dyn-Compression, Web-Http-Errors, Web-Http-Logging, Web-Http-Redirect, Web-Http-Tracing, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Lgcy-Mgmt-Console, Web-Metabase, Web-Mgmt-Console, Web-Mgmt-Service, Web-Net-Ext45, Web-Request-Monitor, Web-Server, Web-Stat-Compression, Web-Static-Content, Web-Windows-Auth, Web-WMI, Windows-Identity-Foundation

From <https://technet.microsoft.com/en-us/library/bb691354{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}28v=exchg.160{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}29.aspx?f=255&MSPPError=-2147217396>

Windows 2016 (Windows Server)

Install-WindowsFeature Net-HTTP-Activation, Desktop-Experience, NET-Framework-45-Features, RPC-over-HTTP-proxy, RSAT-Clustering, RSAT-Clustering-CmdInterface, RSAT-Clustering-Mgmt, RSAT-Clustering-PowerShell, Web-Mgmt-Console, WAS-Process-Model, Web-Asp-Net45, Web-Basic-Auth, Web-Client-Auth, Web-Digest-Auth, Web-Dir-Browsing, Web-Dyn-Compression, Web-Http-Errors, Web-Http-Logging, Web-Http-Redirect, Web-Http-Tracing, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Lgcy-Mgmt-Console, Web-Metabase, Web-Mgmt-Console, Web-Mgmt-Service, Web-Net-Ext45, Web-Request-Monitor, Web-Server, Web-Stat-Compression, Web-Static-Content, Web-Windows-Auth, Web-WMI, Windows-Identity-Foundation

clip_image002


1- (Extending Schema)


Setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>

clip_image003

2- (Preparing AD)


Setup.exe /PrepareAD /OrganizationName:”<organization name>” /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>
Setup.exe /PrepareAD /OrganizationName:TEST /IAcceptExchangeServerLicenseTerms

clip_image004

3- Prepare one domain in the forest or all domains


Setup.exe /PrepareDomain:<FQDN of the domain you want to prepare> /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>

clip_image005
 
To prepare all domains run the following cmd
 
Setup.exe /PrepareAllDomains /IAcceptExchangeServerLicenseTerms
 
From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>
 
Before moving on you will have to fulfill the Software Prerequisites which is attached below.. It’s the same as in Exchange 2013.
 
clip_image007
 
clip_image008
clip_image009
clip_image010
clip_image011
 
Now I will start the Exchange Installation from the Setup, You can follow the setup till the end as in the following screenshots:
 
clip_image001[4]
clip_image002[4]
clip_image003[4]
clip_image004[4]
clip_image005[4]
clip_image006
clip_image007[4]
clip_image008[4]
clip_image009[4]
clip_image010[4]
clip_image011[4]

Now I will create DAG and replicate DBs and notice FastSearch logs


Since this is a LAB and I only have 2 nodes (DAG must have an odd number for Failover), so I am going to use the DC server as my FSW (Which is highly not recommended for Production Environment).

In order for the FSW on DC to work, you will have to add your DC to the Exchange Trusted Subsystem group

clip_image001[6]

Here I added the DC as a member of the group

clip_image002[6]

Normally file server feature is already added to the server by default, but to make sure I’ll run the following command

clip_image003[6]

It’s already there

Now I’ll go back to Exchange servers and add a second NIC for Replication..

I usually rename each NIC so I know which is which, the default NIC belongs to the MAPI traffic and the other one is the replication NIC.

clip_image004[6]

I will configure the Replication NICs on both exchange servers to disable the “Register this connection’s addresses in DNS”

clip_image005[6]
clip_image006[4]

Checking ping between Exchange servers on the Replication NIC

clip_image007[6]

Now I will create the CNO Object (Cluster Name Object) in the Active directory for the DAG

clip_image008[6]
clip_image009[6]

I will disable the object

clip_image010[6]

Double click on the DAG object and go to Security tab and add Exchange servers

clip_image011[6]

Now configure the security for the Exchange members to full

clip_image012

Apply and close…

Now on the DC I will create the FSW’s folder and give it full permission to the Trusted Subsystem group and exchange servers

clip_image013
clip_image014

Click apply and go back to EAC and I’ll start configuring the DAG

clip_image015

Microsoft says that one of the enhancements that have been added to Exchange 2016 is that DatabaseAvailabilityGroupIpAddresses is no longer required when creating a DAG.

By default, the failover cluster will be created without an administrative access point, as this is the recommended best practice.

From <http://blogs.technet.com/b/exchange/archive/2015/05/05/exchange-server-2016-architecture.aspx>

So in this case we won’t need to assign any IP address to the DAG…

clip_image016

I’ll click on save and see what happens

clip_image017

Navigating to the administrators group on AD, The Exchange subsystem group is not added so I’ll add it.

clip_image018
clip_image019

Now I will add Exchange servers as members to the DAG

clip_image020
clip_image021
clip_image022

Upon adding the Exchange members to DAG I got the following error

A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2K16.test.com’.. [Server: EXCH2K16.test.com]
error
A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2k1602.test.com’.. [Server: EXCH2k1602.test.com]

clip_image023

So I checked the following,
1- Firewall
2- CNO’s security settings.

Error occurs due to firewall being enabled on the DC (Where the FSW is )

clip_image024

I disabled the firewall and gave full permission to the Exchange trusted system to the DAG object

clip_image025

After that I signed out of Exchange servers, signed back in.. Deleted DAG and recreated it… that didn’t work either

Tried using Exchange management shell but it didn’t work too

clip_image026
clip_image027

Checking the log coming in the description, I find out the log is complaining about not finding DAG while trying to resolve it.

clip_image028

Also the log says that it has installed Failover cluste rbut still the cluster can’t find FQDN called DAG.

So I will have to configure DAG in the dns and give it an IP of my first Exchange server

clip_image029
clip_image030

Checking DAG resolving from Exchange server

clip_image031

As the log says, restart is required after installing failover cluster so I’ll restart Exchange servers and then retry to add Exchange servers to the dag.

After restarting the server, It seems that things are working

clip_image032
clip_image033

The second server gave the following error


The Microsoft Exchange Replication service does not appear to be running on “EXCH2k1602”. Make sure that the server is operating, and that the services can be queried remotely.

Apparently the error is correct, After restarting the server it turned out that most of the second Exchange server’s services were not working..

To be honest I didn’t ask myself why did not the services started since I am using a preview version of Exchange 2016 and Windows as well so I manually started all the services.

clip_image034

Interestingly while checking services, I noticed new services e.g. (DAG Management, Compliance Audit, Notifications broker)

After starting the services, now I tried to add the second server again to the DAG.

clip_image035
clip_image036

So eventually, DAG doesn’t need an IP address but still a DNS value needs to be created for the NCO object and needs to have an IP assigned to it which will be the Exchange server IP address..

clip_image037

Next: I will add a database copy and see how it’s improved and do I need to restart the IS service as in Exchange 2013.

clip_image038
clip_image039

I will leave all the default values and add the second server for the database to be copied on. Unlike Exchange 2013 in most of the times the database would fail first and gives an error ..

In 2016 it starts directly seeding the database to the second Exchange server that’s member of the DAG.

clip_image040
clip_image041
clip_image042
 
On the second Server where the database has been copied to, I checked the Logs and Fastsearch was throwing errors as usual since database logs are not copied … as soon as the database logs finished copying the fastsearch will return ok and the database will appear as Healthy in EAC.
 
clip_image043

Fastsearch finally reported that indexing started on the newly copied DB.

clip_image044
The database copy should now report healthy in the EAC.
clip_image045
 
Hope you find this useful.
In the next article I will publish the Exchange server online to check the rest of the functionalities.

Export and import PST to Office 365 Exchange online mailboxes

Microsoft has launched a new feature that allows administrators to import PST to Exchange online directly through the portal.

 

In this article I’ll guide you through the steps of uploading one PST file and import it to a user’s mailbox. Although the steps are identical to Microsoft’s TechNet article but it’s more detailed and with screenshots. As I was personally reading and following the article at some point I got confused as to what which shared folder is the article speaking of and little tiny bits that are not detailed since Microsoft combines two methods in the same page “Ship data on Physical Drives and Upload files over the network”

 

So to achieve this, you’ll have to first sign in to your Office 365 portal. Open Exchange admin center and follow the below steps:

 

  1. Granting Permission

 

Grant your self-importing PST permission to users by navigating to Exchange admin center -> Permissions> Double click on Compliance Management

Under Roles: click on + and add Mailbox Import Export role

Click on + Under Members and add your user account

clip_image001[5]

 

2. `Copy Secure URL and secure storage account key

 

To get the Azure secure storage account key and URL you will have to go back to the Office 365 portal and then click on Import tab on the left pane

Then click on the Key sign below

 

clip_image002[4]

 

When you click on it, you will be able to retrieve the key and the URL by clicking on Copy Key and URL .

 

clip_image003[4]

 

The copy is pretty long and you’ll have to notice that sometimes you might get confused and copy only the appearing portion of it in the field… if you do so and copied that in the Azcopy command or Azure storage explorer you might get an error …

Here’s my Secure Storage account key that I am using on a trial version of Office 365.

 

KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g==

 

The URL has an important part which you will be using in Azure Storage Explorer tool in order to login and browse your Tenant’s storage which you’ll use to upload PST to.

The URL will appear as following.. You will need to copy the part highlighted in bold

https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/

 

You have to copy this in red in to the storage account name

d49d7ae0e38a4d8e9c93565

 

3. Copying PST files to Azure Folder using Azcopy command or Azure Storage Explorer (You can use Azure Storage Explorer too)

 

In order to upload PST files to Azure, you have two methods. The first is using Azcopy command which is pretty easy and straightforward (but still CMD dependent) or you can use the GUI Application which is Azure Storage explorer

To download azcopy, you can use the following link

 

http://az635501.vo.msecnd.net/azcopy-3-2-0/MicrosoftAzureStorageTools.msi

 

Or download them from the Import page as well under Resources:

 

clip_image004[4]

 

Once the tool is installed. Right click on it and open it as administrator

 

The following command will take all the files inside my local folder path C:UsersMohammedDesktopupload

It will create a folder in Azure’s default folder ingestiondata called “Server01/PSTshareR1/”

 

It will use the destkey that I have retrieved from Office 365 Import window. And will leave all the logs in your local drive c:PSTuploadUploadlog.log

 

AzCopy /Source:C:UsersMohammedDesktopupload /Dest:https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/SERVER01/PSTshareR1/

/Destkey:KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g== /S /V:C:PSTUploadUploadlog.log

 

clip_image005[4]

 

To make sure that files are uploaded. I will open Azure Storage Explorer 6 (Preview) and click Add Account on top

On add storage account window I will use the blob name that I have got from the URL earlier and storage secure key in the storage account key below and click on save.

 

clip_image006[4]

 

Once I click that I will get a list of directories .. The default directory which is used by Office 365 is the “Ingestiondata” folder .. There our files will be uploaded.

 

clip_image007[4]

 

https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

 

4. Create CSV File to import PST

 

Assuming you have 150 PST files that you want to upload and import into users which already have been enabled on Exchange online … In order to do so you will have to prepare a CSV file that looks like the below sample

 

To provide an explanation of what each column stands for .. Microsoft has written a table that clears the dust but some parts were not even clear for me like the FilePath as in the TechNet article it gets you confused with the “Ship data on Physical hard drives” since it uses your drive to upload data directly to Azure through the Import tool on Office 365 portal.

 

image

 

From <https://technet.microsoft.com/library/ms.o365.cc.IngestionHelp.aspx?v=15.1.166.0&l=1&f=255&MSPPError=-2147217396>

Note:

The friendly path here is the path of the folder you have created in Azure through the Azcopy command

AzCopy /Source:C:UsersMohammedDesktopupload /Dest:https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/SERVER01/PSTshareR1/

/Destkey:KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g== /S /V:C:PSTUploadUploadlog.log

 

clip_image008[4]

 

So the CSV File is ready.

 

 

In Azure Storage Explorer I doubled check if the PST files has finished uploading and it’s there.

clip_image009[4]

 

5. Using the Upload Files over the network

 

Back to Office 365 portal, go to Import and click on the + Sign and select Upload files over the network

 

clip_image010[4]

 

Select I have access to the mapping file as well

 

clip_image011[4]

 

Click on + and upload the CSV file that you have prepared for the mapping

Next File is imported, Click on “By checking this box, you agree to the terms and conditions of this service.

 

clip_image012[4]

 

As soon as you accept and click next the Import is going to check path, email, folder and will start the import process.

 

clip_image013[4]

clip_image014[4]

 

Email before importing

 

clip_image015[4]

 

Imported started, folder has been created

 

clip_image016[4]

 

Importing is done

 

clip_image017[4]

clip_image018[4]

 

Importing is done

Reference

 

https://technet.microsoft.com/library/ms.o365.cc.IngestionHelp.aspx?v=15.1.166.0&l=1&f=255&MSPPError=-2147217396#BKMK_CreateAnewMappingtoupload

https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

Exchange 2013 ECP (Admin Panel Page) Fails with 500 Unexpected error after running Hybrid Configuration Wizard with Office 365

Symptoms

After you attempt to try Hybrid Configuration Wizard between Exchange 2013 SP1 and Exchange online (Office 365) .. You are unable to login to your OWA/ECP Page and instead you get an error:

 

clip_image001

 

Once you look at your event viewer you notice it records the event ID 4 with the following error

 

clip_image002

 

Cause:

The HCW or “Hybrid Configuration Wizard” In Exchange 2013 (CU6 or 8) might cause some changes to your CAS folder in the following path and file

“c:Program FilesMicrosoftExchange ServerV15ClientAccessecpDDI”` the file name is RemoteDomains.xaml and multiply some variables which causes ECP to fail and report that error.

 

Error:

Current user: ‘Domain.local/User’

Request for URL ‘https://ex2k1301.Domain.local:444/ecp/default.aspx(https://mail.Domain.com/ecp/)‘ failed with the following error:

System.Web.HttpUnhandledException (0x80004005): Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Data.DuplicateNameException: A column named ‘TargetDeliveryDomain’ already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

— End of inner exception stack trace —

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Data.DuplicateNameException: A column named ‘TargetDeliveryDomain’ already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

— End of inner exception stack trace —

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

System.Data.DuplicateNameException: A column named ‘TargetDeliveryDomain’ already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

Flight info: Features:[[Global.DistributedKeyManagement, False],[Global.GlobalCriminalCompliance, False],[Global.MultiTenancy, False],[Global.WindowsLiveID, False],[Eac.AllowMailboxArchiveOnlyMigration, True],[Eac.AllowRemoteOnboardingMovesOnly, False],[Eac.BulkPermissionAddRemove, True],[Eac.CmdletLogging, True],[Eac.CrossPremiseMigration, False],[Eac.DevicePolicyMgmtUI, False],[Eac.DiscoveryDocIdHint, False],[Eac.DiscoveryPFSearch, False],[Eac.DiscoverySearchStats, False],[Eac.DlpFingerprint, False],[Eac.EACClientAccessRulesEnabled, False],[Eac.GeminiShell, False],[Eac.ManageMailboxAuditing, False],[Eac.ModernGroups, False],[Eac.Office365DIcon, False],[Eac.OrgIdADSeverSettings, False],[Eac.RemoteDomain, False],[Eac.UCCAuditReports, False],[Eac.UCCPermissions, False],[Eac.UnifiedAuditPolicy, False],[Eac.UnifiedComplianceCenter, False],[Eac.UnifiedPolicy, False],[Eac.UnlistedServices, False],], Flights:[], Constraints:[[LOC, EN-US],[MACHINE, EX2K1301],[MODE, ENTERPRISE],[PROCESS, W3WP],[USER, Username(EditeD)@],[USERTYPE, BUSINESS],], IsGlobalSnapshot: False

 

Solution:

Looking in the path I have wrote above “MicrosoftExchange ServerV15ClientAccessecpDDI” and opening the file that I have mentioned “RemoteDomains.xaml” you can clearly see there are incorrect format that have been duplicated 3 times.

clip_image003

To make sure that this was the cause, I have another server with CU8 on it so I went and checked the same file which was in the same location to find the result different.

This is the server that works in another environment and doesn’t have any issue.

clip_image004

So, the solution was to remove the two duplicates and correct the format of the variable line… I corrected the first line that includes

<Variable DataObjectName=”RemoteDomain” Name=”TargetDeliveryDomain” Type=”{x:Type s:Boolean}” />

And deleted the other two identical lines.. then I saved the file and closed notepad.

clip_image005

Next: Open IIS on the same server and go to “Application Pools” right click on the affected pools and Recycle them.. You don’t need to reset IIS as the fix should work right away.

Recycle ECP Pool

clip_image006

After recycling checking if the pool is reporting started or not…

clip_image007

Next I open ECP and it works without an issue

clip_image008

 

Hope this helps Winking smile

Prepare Schema for Exchange 2013 Migration while having Hybrid Integration with Exchange 2010

 

In a very interesting situation that I came through I had an environment with two DCs and Exchange 2010 that I had previously setup for Hybrid integration with Office 365 for demonstration with a trial subscription but I haven’t removed the integration after I finished my test and the trial expired and the tenant was deleted.

Next I intended to upgrade my existing Exchange 2010 to Exchange 2013 and setup coexistence between them however, I have stumbled in the step of preparation of AD schema for Exchange 2013. While trying to prepare the schema I got the following error

clip_image001

Setup /PrepareSchema /IAcceptExchangeServerLicenseTerms

Welcome to Microsoft Exchange Server 2013 Cumulative Update 8 Unattended Setup

Copying Files…

File copy complete. Setup will now collect additional information needed for

installation.

Performing Microsoft Exchange Server Prerequisite Check

Prerequisite Analysis FAILED

A hybrid deployment with Office 365 has been detected. Please ensure that you are running setup with the /TenantOrganizationConfig switch. To use the TenantOrganizationConfig switch you must first connect to your Exchange Online tenant via PowerShell and execute the following command: “Get-OrganizationConfig | Export-Clixml -Path MyTenantOrganizationConfig.XML”. Once the XML file has been generated, run setup with the TenantOrganizationConfig switch as follows “/TenantOrganizationConfig MyTenantOrganizationConfig.XML”.

If you continue to see this this message then it indicates that either the XML file specified is corrupt, or you are attempting to upgrade your on-premises Exchange installation to a build that isn’t compatible with the Exchange version of your Office 365 tenant. Your Office 365 tenant must be upgraded to a compatible version of Exchange before upgrading your on-premises Exchange installation. For

more information, see: http://go.microsoft.com/fwlink/?LinkId=262888

For more information, visit: http://technet.microsoft.com/library(EXCHG.150

)/ms.exch.setupreadiness.DidTenantSettingCreatedAnException.aspx The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the <SystemDrive>:ExchangeSetupLogs folder.

 

The Office 365 Hybrid setup was still there in my Exchange Console and since I couldn’t follow MS’s recommended steps to connect to O365 tenant and get the XML file then I had to do things manually.

First I connected to the EMC and removed all the instances that were created during the Exchange Hybrid Wizard Configuration

1- Organization Relationships

clip_image001[4]

 

image

 

2- Federation Trust

clip_image001[6]

clip_image002

 

3- Remote Domains

 

clip_image001[8]

clip_image002[4]

 

4- Accepted Domains

clip_image001[10]

clip_image002[6]

 

5- Send and Receive Connectors

clip_image001[12]

clip_image002[8]

clip_image003

 

Lastly the Hybrid Configuration object…

Since remove-hybridconfiguration cmdlet is not supported to remove the hybrid configuration object from AD then we have no choice but to use ADSIEDIT tool to do so.

I will navigate to Configuration > Services > Microsoft Exchange > First Organization > Delete “CN=Hybrid Configuration”

 

image

 

image

image

 

Restart MSExchangeServicehost

clip_image001[14]

 

image

 

Now I will try again to prepare AD schema for Exchange 2013 but I got a different error

clip_image001[16]

Extending Active Directory schema FAILED

The following error was generated when “$error.Clear);

install-ExchangeSchema -LdapFileName ($roleInstallPath + “SetupData”+$

RoleSchemaPrefix + “schema0.ldf”)” was run: “Microsoft.Exchange.Configuration.Tasks.TaskException: There was an error while running ‘ldifde.exe’ to import the schema file ‘C:WindowsTempExchangeSetupSetupDataPostExchange2003_schema0.ldf’. The error code is: 8224. More details can be found in the error file: ‘C:UsersAdministrator.DEMOTESASAppDataLocalTemp2ldif.err’at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.ImportSchem

aFile(String schemaMasterServer, String schemaFilePath, String macroName, StringmacroValue, WriteVerboseDelegate writeVerbose)at Microsoft.Exchange.Management.Deployment.InstallExchangeSchema.InternalPro

cessRecord()at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__b()at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)”.The Exchange Server setup operation didn’t complete. More details can be found

in ExchangeSetup.log located in the <SystemDrive>:ExchangeSetupLogs folder.

 

Checking the ldif.err file mentioned in the error above it seems that Exchange is complaining because the changes of the AD schema is not being replicated to the other AD partners which is true since I have another additional dC that’s turned off.

clip_image001[19]

After turning on the other DC we’ll see what happens

The other DC had another issue as I have turned it off for long time and it was not syncing due to expired Tomb stone life so I had to fix this issue as well and I have published it in a different article.

Please click here to see how the replication issue was fixed.

 

Issue has been fixed

clip_image001[21]

 

clip_image002[10]

 

Hope someone finds this useful Winking smile