Category Archives: Microsoft Exchange

Exchange 2016, Exchange 2019, Microsoft Exchange, Office365

Search and Delete certain Items/Folders from a Mailbox

December 16, 2019 Mohammed Hamada Leave a comment

The Story

During a project of Hybrid migration from Exchange on-premises to Exchange online, I was almost about to finalize the project by moving the last remaining users mailboxes however had an interesting issue to deal with where a user was failing with the following error:

Error:

Error: MigrationPermanentException: Mailbox dumpster size 50.87 GB (54,620,074,576 bytes) exceeds target quota 30 GB –> Mailbox dumpster size 50.87 GB exceeds target quota.

After some research it turned out that you can clean the dumpster using search-mailbox PowerShell cmdlet, Sync the user’s object with ADConnect and then continue the migration from the last failure.

To solve the issue, Go on your Exchange on-premises and launch Exchange Management shell

Solution:

First, Let’s see the user’s dumpster and recoverable items

Get-MailboxFolderStatistics -Identity “User” -FolderScope RecoverableItems | Format-Table Name,FolderPath,ItemsInFolder,FolderAndSubfolderSize

To Delete the dumpster only use this

Delete dumpster only

Search-mailbox -identity User -SearchDumpsterOnly –DeleteContent

To delete a certain email with certain subject in the dumpster use the following:

Get-mailbox “user”| search-mailbox –searchquery “Subject:’*'” –DeleteContent –SearchDumpsterOnly

The cmdlet will search and delete

Reference:

https://docs.microsoft.com/en-us/microsoft-365/compliance/search-for-and-delete-messagesadmin-help

Exchange 2019, Microsoft Exchange, Office 365

MRSPROXY.SVC’FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT. THE REMOTE SERVER RETURNED AN ERROR: (404) NOT FOUND

December 16, 2019 Mohammed Hamada Leave a comment

Symptoms

MRSPROXY.SVC’FAILED BECAUSE NO SERVICE WAS LISTENING ON THE SPECIFIED ENDPOINT. THE REMOTE SERVER RETURNED AN ERROR: (404) NOT FOUND

You get an error when you’re trying to setup Hybrid configuration between your Exchange On-premises and Exchange Online.

After I had one issue like this I did some research and used Fiddler/Wireshark to check for traffic I noticed that Exchange was not encrypting traffic and testing the Migration Server Availability was reporting that the MRS service was not listening on the supposed port which is 443.

In this case the solution was pretty easy, but still you’ll have to make a hard choice of choosing to place Exchange behind a load balancing with SSL Offloading on or not.

In my case I had to turn off the SSL Offloading on the Load balancer and that alone was enough to get this working.

Resolution:

Make sure that SSL Offloading is disabled on OWA/OA and Load balancer if there’s one.

Other resolutions:

https://support.microsoft.com/en-us/kb/3065754

Cloud, Microsoft Exchange, Office 365, Office365, SMTP

Installing and configuring IIS SMTP for Office 365 with User License

September 6, 2019 Mohammed Hamada Leave a comment

Prerequisites:

1- Office 365 Licensed User.

2- Windows Server 2016 OS.

3- Public Static IP (To add to the SPF policy to avoid getting emails flagged as spam)

4- Allow outbound port 587 on your Firewall.

Regardless of Windows Operating system you’re going to install the SMTP Role the steps are the same. You’re going to install the SMTP Server feature from the Server Manager as following:

First Launch Server Manager (Windows 2016) and click on Manage\Add Roles and Features

Click next to choose the installation type, Make sure you choose the Role-based or Feature type of installation

Next select the server where you want to install the SMTP Feature

Click next to head to the next step, The SMTP feature is normally located in the Features page not the role so proceed further by clicking Next

Scroll down in the Feature page until you get to SMTP Server

Click on the checkbox next to SMTP Server and add the features which will automatically pop up

Click Next

Click Next again, Leave unchanged here

Click Install and you will get your SMTP Server ready

Open IIS 6.0 Manager to Manage your Relay

To setup SMTP server for relay through Office 365 follow those steps:

Set up SMTP

Open Server Manager, select Tools, and then select Internet Information Services (IIS) 6.0.

– Expand the current server, right-click the SMTP Virtual Server, and then select Properties.
– On the General tab, select Advanced > Add.
– In the IP Address box, specify the address of the server that’s hosting the SMTP server.
– In the Port box, enter 587 and select OK.
– On the Access tab, do the following:

On the Access tab, do the following:

Select Authentication and make sure that Anonymous Access is selected.

Select Connection > Only the List Below, and then specify the IP addresses of the devices that will be connecting to the SMTP server, such as printers.

Select Relay > Only the List Below, and then specify the IP address of the devices relaying through this SMTP server

On the Delivery tab, select Outbound Security, and then do the following:
1. Select Basic Authentication.
2. Enter the credentials of the Office 365 user who you want to use to relay SMTP mail.
3. Select TLS Encryption.
4. Select Outbound Connections, and in the TCP Port box, enter 587 and select OK.
5. Select Advanced and specify SMTP.office365.com as the Smart Host.

Restart the IIS service and the SMTP service.

https://technet.microsoft.com/en-us/library/dn592151v=exchg.150).aspx>

Let’s make a test by creating a notepad file with the following Test

Ref:

https://technet.microsoft.com/en-us/library/dn592151(v=exchg.150).aspx

Microsoft, Microsoft Exchange, Security

Russian cyberspies are using one hell of a clever Microsoft Exchange backdoor

May 7, 2019 Mohammed Hamada Leave a comment

Turla APT found exploiting LightNeuron backdoor, a first of its kind targeting Microsoft Exchange email servers.

A Russian cyber-espionage group has developed and has been using one of the most complex backdoors ever spotted on an email server, according to new research published today by cyber-security firm ESET.

The backdoor, named LightNeuron, was specifically designed for Microsoft Exchange email servers and works as a mail transfer agent (MTA) –an approach that no other backdoor has ever taken.

“To our knowledge, this is the first malware specifically targeting Microsoft Exchange,” ESET Malware Researcher Matthieu Faou told ZDNet via email.

“Turla targeted email servers in the past using a malware called Neuron (a.k.a DarkNeuron) but it was not specifically designed to interact with Microsoft Exchange.

“Some other APTs use traditional backdoors to monitor mail servers’ activity. However, LightNeuron is the first one to be directly integrated into the working flow of Microsoft Exchange,” Faou told us.

Because of the deep level the backdoor works, LightNeuron allows hackers to have full control over everything that passes through an infected email server, having the ability to intercept, redirect, or edit the content of incoming or outgoing emails.

LIGHTNEURON DEVELOPED BY TURLA GROUP

This makes LightNeuron one of the most powerful tools of its kind, and a tool fit to be in the arsenal of Turla, one of the world’s most advanced nation-state hacking units.

The Turla APT (advanced persistent threat) is infamous for past operations that seem to be pulled out of Hollywood movies. The group has been known to hijack and use telecommunications satellites to deliver malware to remote areas of the globe, has developed malware that hid its control mechanism inside comments posted on Britney Spears’ Instagram photos, and has hijacked the infrastructure of entire ISPs to redirect users to malware.

In a report released today, ESET says that Turla has been using LightNeuron for almost five years, since 2014, which again shows the tool’s advanced capabilities, being able to avoid detection for so many years.

To be fair, the first mention of LightNeuron was in a Kaspersky Lab report on the APT Trends of Q2 2018. However, Kaspersky only described the tool in brief. The ESET report released today shines more light on the tool’s unique capabilities that make it stand out from all other backdoors deployed on email servers up until now.

Researchers warn that LightNeuron is currently being used in live attacks and that Turla also appears to have created a UNIX port –which ESET hasn’t been able to find until now.

The Slovak cyber-security firms said it detected three victim organizations infected with Turla’s LightNeuron backdoor. The company did not name the victims, but provided general descriptions:

– Unknown organization in Brazil
– Ministry of Foreign Affairs in Eastern Europe
– Regional diplomatic organization in the Middle East

A CLEVER WAY OF CONTROLLING LIGHTNEURON

According to researchers, the thing that made LightNeuron stand out, besides being the first backdoor for Microsoft Exchange servers, was its command-and-control mechanism.

Once a Microsoft Exchange server is infected and modified with the LightNeuron backdoor, hackers never connect to it directly. Instead, they send emails with PDF or JPG attachments.

Using the technique of steganography, Turla hackers hide commands inside PDF and JPG images, which the backdoor reads and then executes.

Per ESET, LightNeuron is capable of reading and modifying any email going through the Exchange server, composing and sending new emails, and blocking a user from receiving certain emails.

Furthermore, victim organizations will have a hard time detecting any interactions between Turla operators and their backdoor, mainly because the commands are hidden inside PDF/JPG code and the incoming emails could be disguised as banal spam.

In addition, if anyone had any doubts LightNeuron was the work of Russian hackers, ESET researchers said that in the cases they investigated they found that Turla operators only sent commands to backdoored servers during a typical 9-to-5 workday in the UTC+3 (Moscow) timezone, and took a break from all operations between December 28, 2018, and January 14, the typical Christmas and New Year holidays for Eastern Orthodox Christians –Russia’s main religion.

Image: ESET

Because LightNeuron works at the deepest levels of a Microsoft Exchange server, removing this backdoor is quite problematic.

ESET released a white paper today with detailed removal instructions.

Exchange 2010, Exchange 2013, Exchange 2016, Microsoft Exchange, Powershell

Useful Powershell script to resolve the X500 address

April 9, 2018 Mohammed Hamada Leave a comment

In migration, Powershell can be a very crucial tool to achieve success and finalize projects within deadline or even fix issues.

During the time of working with Exchange we had lots of issues with users not able to send an email to their migrated colleagues due to some issues with contacts which was caused by the Legacy Exchange DN not being migrated with the user or lost due to some wrong deletion.

Once users try to send an email to that particular user with the missing Legacy Exchange DN. The receiving Exchange server will result an error and send it to the user as NDR message explaining to them that the error is due to not finding the particular address.

The solution to this particular problem is very simple especially if it’s couple of users however to resolve the address you’ll need to google and understand the language that Exchange server users to match the original used address in the missing user’s attributes.

The below script would work accordingly with whatever situation that faced me and it became very handy to me.

How to use:

1- Copy the script to a notepad and save as convert.ps1 on Desktop

2- Run script and try to type in powershell convert-X500 then hit enter.

3- Copy and paste the address you got from the error message above.

Once you copy and paste hit enter and you’ll get the final result

Note: Make sure you remove the @domain.local in the end

Function Convert-X500{ # Define the Legacy Exchange DN here
Write-Host “”Enter your X500 Address here…”” -ForegroundColor Green -BackgroundColor Black
$X500Source = read-host

# Converts the various strings to the proper syntax
$X500 = $X500Source.Replace(“_”, “/”)
$X500 = $X500.Replace(“+20″, ” “)
$X500 = $X500.Replace(“IMCEAEX-“, “”)
$X500 = $X500.Replace(“+28”, “(“)
$X500 = $X500.Replace(“+29”, “)”)
$X500 = $X500.Replace(“+2E”, “.”)
$X500 = $X500.Replace(“+5F”, “_”)
$X500 = $X500.Replace(“@YourDC.localHere“, “”)

Write-Host X500:$X500}

Exchange 2013, Hosted, Microsoft Exchange

Hosted Exchange 2013 Tenant setup step by step

January 17, 2017 Mohammed Hamada Leave a comment

Starting with Exchange 2013 Hosted .. this is a brief article about how to setup Tenants correctly step by step

1- Create OU

Create New-ADOrganizationalUnit -Name Test10 -Path ou=test10,OU=ExchHosting,DC=lab,DC=com

2- Set OU’s UPN

set-adforest -identity exch01.lab.com -UPNSuffixes @{add=”test10.com”}

Moving to Exchange Management Shell

3- create accepted domain on Exch

New-AcceptedDomain -Name “test10” -DomainName test10.com -DomainType:authoritative

4- Create address book

New-GlobalAddressList -Name “test10 – GAL” -ConditionalCustomAttribute1 “test10” -IncludedRecipients MailboxUsers -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

Note:

Address books once created is virtually separated (That means you don’t have to do anything to separate Address books unless you want them to see each other)

https://technet.microsoft.com/en-us/library/hh529948(v=exchg.150).aspx

If needed then create All Rooms Address List

New-AddressList -Name “Test10 – All Rooms” -RecipientFilter “(CustomAttribute1 -eq ‘test10’) -and (RecipientDisplayType -eq ‘ConferenceRoomMailbox’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

6- And time to create All Users Address List

New-AddressList -Name “Test10 – All Users” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘User’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

7- The All Contacts Address List

New-AddressList -Name “Test10 – All Contacts” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘Contact’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

8-The All Groups Address List

New-AddressList -Name “Test10 – All Groups” -RecipientFilter “(CustomAttribute1 -eq ‘Test10’) -and (ObjectClass -eq ‘Group’)” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com”

9-Now for something quite useful: Offline Address Book

New-OfflineAddressBook -Name “Test10” -AddressLists “Test10 – GAL”

Mail address policy

New-EmailAddressPolicy -Name “Test10 – EAP” -RecipientContainer “ou=test10,OU=ExchHosting,DC=lab,DC=com” -IncludedRecipients “AllRecipients” -ConditionalCustomAttribute1 “Test10” -EnabledEmailAddressTemplates “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”,”smtp:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}m@test10.com” -EnabledPrimarySMTPAddressTemplate “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”

Set current enabled mail in the test10 organization’s smtp address to test.com

Set-EmailAddressPolicy -Identity “Test10 – EAP” -EnabledPrimarySMTPAddressTemplate “SMTP:{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}g.{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}s@test10.com”

If you’re connected from the Normal Exchange management shell you’ll normally get an error stating that users don’t exist on your Active Directory which is normal since the session you’re connected to is connected to the top forest users and can’t see the address book that’s isolated from other tenants.

You will need to open Powershell (not Exchange management Shell) and use the following cmdlet

Let’s connect to exchange, but before we have to add one of our Organization test10 users to the Organization Management Group in order to be able to administer his organization

$cred = get-credential

$Session = New-PSSession -Authentication basic -Credential $cred -ConnectionUri https://mr.moh10ly.website/PowerShell/ -ConfigurationName Microsoft.Exchange -AllowRedirection

Import-PSSession $Session

11- Address book policy

Now we need to create the Address Book Policy

New-AddressBookPolicy -Name “Test10” -AddressLists “Test10 – All Users”, “Test10 – All Contacts”, “Test10 – All Groups” -GlobalAddressList “Test10 – GAL” -OfflineAddressBook “Test10” -RoomList “Test10 – All Rooms”

12- Optional

New-Mailbox -Name ‘Test10 Conference Room 1’ -Alias ‘test10_conference’ -OrganizationalUnit ‘ou=test10,OU=ExchHosting,DC=lab,DC=com’ -UserPrincipalName ‘test10conf1@test10.com’ -SamAccountName ‘test10_conference’ -FirstName ‘Conference’ -LastName ‘Room 1’ -AddressBookPolicy ‘Test10’ -Room

To mark this conference room for the tenant test10 I’ll have to use this cmdlet otherwise it won’t show in the address book of test 10

Set-Mailbox test10_conference -CustomAttribute1 ‘Test10’

Set calendar settings for the conference room we have just created as following

Set-CalendarProcessing -Identity test10_conference -AutomateProcessing AutoAccept -DeleteComments $true -AddOrganizerToSubject $true -AllowConflicts $false

13- Create mailboxes

$c = Get-Credential

New-Mailbox -Name ‘Mohammed Test’ -Alias ‘test10_mohammed’ -OrganizationalUnit ‘ou=test10,OU=ExchHosting,DC=lab,DC=com’ -UserPrincipalName ‘mohammed@test10.com’ -SamAccountName ‘test10_mohammed’ -FirstName ‘Mohammed’ -LastName ‘HAmada’ -Password $c.password -ResetPasswordOnNextLogon $false -AddressBookPolicy ‘Test10’

Now once I use this script the user will be created and you’ll be able to see In from the ECP

But you won’t be able to see the user in the address book of the tenant since we didn’t apply the tenant which this user belongs to

To do so we’ll have to use the following CMDLET

Set-Mailbox arthas@test10.com -CustomAttribute1 “Test10”

Exchange 2016, Microsoft Exchange, Windows Server

Preparing, Installing and Configuring Exchange 2016 with DAG on Windows 2016

August 7, 2015 Mohammed Hamada 2 Comments

So Exchange 2016 preview version came on MSDN and I decided to give it a try along with the DAG …

Previously in Exchange 2013 I used to have an issue with the fast search on DAG as in some cases it used to stop and cause the original database and copy to report not healthy.

Here I wanted to Install exchange 2016 on new windows edition along with configuring DAG and observe the database’s indexing status.

So to start, I used the available Microsoft Technet related to Exchange 2016.

I’ll launch Powershell as adminsitrator an start by installing the requested software

Install-WindowsFeature RSAT-ADDS

From <https://technet.microsoft.com/en-us/library/bb691354{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}28v=exchg.160{308b10a016e19a1cd6a208cbc3961927e16fc6766a4020d3c4ef54ea17925f0f}29.aspx?f=255&MSPPError=-2147217396>

Windows 2012/2012R2

Install-WindowsFeature AS-HTTP-Activation, Desktop-Experience, NET-Framework-45-Features, RPC-over-HTTP-proxy, RSAT-Clustering, RSAT-Clustering-CmdInterface, RSAT-Clustering-Mgmt, RSAT-Clustering-PowerShell, Web-Mgmt-Console, WAS-Process-Model, Web-Asp-Net45, Web-Basic-Auth, Web-Client-Auth, Web-Digest-Auth, Web-Dir-Browsing, Web-Dyn-Compression, Web-Http-Errors, Web-Http-Logging, Web-Http-Redirect, Web-Http-Tracing, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Lgcy-Mgmt-Console, Web-Metabase, Web-Mgmt-Console, Web-Mgmt-Service, Web-Net-Ext45, Web-Request-Monitor, Web-Server, Web-Stat-Compression, Web-Static-Content, Web-Windows-Auth, Web-WMI, Windows-Identity-Foundation

Windows 2016 (Windows Server)

Install-WindowsFeature Net-HTTP-Activation, Desktop-Experience, NET-Framework-45-Features, RPC-over-HTTP-proxy, RSAT-Clustering, RSAT-Clustering-CmdInterface, RSAT-Clustering-Mgmt, RSAT-Clustering-PowerShell, Web-Mgmt-Console, WAS-Process-Model, Web-Asp-Net45, Web-Basic-Auth, Web-Client-Auth, Web-Digest-Auth, Web-Dir-Browsing, Web-Dyn-Compression, Web-Http-Errors, Web-Http-Logging, Web-Http-Redirect, Web-Http-Tracing, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Lgcy-Mgmt-Console, Web-Metabase, Web-Mgmt-Console, Web-Mgmt-Service, Web-Net-Ext45, Web-Request-Monitor, Web-Server, Web-Stat-Compression, Web-Static-Content, Web-Windows-Auth, Web-WMI, Windows-Identity-Foundation

1- (Extending Schema)

Setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>

2- (Preparing AD)

Setup.exe /PrepareAD /OrganizationName:”<organization name>” /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>
Setup.exe /PrepareAD /OrganizationName:TEST /IAcceptExchangeServerLicenseTerms

3- Prepare one domain in the forest or all domains

Setup.exe /PrepareDomain:<FQDN of the domain you want to prepare> /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>

To prepare all domains run the following cmd

Setup.exe /PrepareAllDomains /IAcceptExchangeServerLicenseTerms

From <https://technet.microsoft.com/en-us/library/bb125224(v=exchg.160).aspx#Step1>

Before moving on you will have to fulfill the Software Prerequisites which is attached below.. It’s the same as in Exchange 2013.

Now I will start the Exchange Installation from the Setup, You can follow the setup till the end as in the following screenshots:

Now I will create DAG and replicate DBs and notice FastSearch logs

Since this is a LAB and I only have 2 nodes (DAG must have an odd number for Failover), so I am going to use the DC server as my FSW (Which is highly not recommended for Production Environment).

In order for the FSW on DC to work, you will have to add your DC to the Exchange Trusted Subsystem group

Here I added the DC as a member of the group

Normally file server feature is already added to the server by default, but to make sure I’ll run the following command

It’s already there

Now I’ll go back to Exchange servers and add a second NIC for Replication..

I usually rename each NIC so I know which is which, the default NIC belongs to the MAPI traffic and the other one is the replication NIC.

I will configure the Replication NICs on both exchange servers to disable the “Register this connection’s addresses in DNS”

Checking ping between Exchange servers on the Replication NIC

Now I will create the CNO Object (Cluster Name Object) in the Active directory for the DAG

I will disable the object

Double click on the DAG object and go to Security tab and add Exchange servers

Now configure the security for the Exchange members to full

Apply and close…

Now on the DC I will create the FSW’s folder and give it full permission to the Trusted Subsystem group and exchange servers

Click apply and go back to EAC and I’ll start configuring the DAG

Microsoft says that one of the enhancements that have been added to Exchange 2016 is that DatabaseAvailabilityGroupIpAddresses is no longer required when creating a DAG.

By default, the failover cluster will be created without an administrative access point, as this is the recommended best practice.

From <http://blogs.technet.com/b/exchange/archive/2015/05/05/exchange-server-2016-architecture.aspx>

So in this case we won’t need to assign any IP address to the DAG…

I’ll click on save and see what happens

Navigating to the administrators group on AD, The Exchange subsystem group is not added so I’ll add it.

Now I will add Exchange servers as members to the DAG

Upon adding the Exchange members to DAG I got the following error

A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2K16.test.com’.. [Server: EXCH2K16.test.com]
error
A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering isn’t installed on ‘EXCH2k1602.test.com’.. [Server: EXCH2k1602.test.com]

So I checked the following,
1- Firewall
2- CNO’s security settings.

Error occurs due to firewall being enabled on the DC (Where the FSW is )

I disabled the firewall and gave full permission to the Exchange trusted system to the DAG object

After that I signed out of Exchange servers, signed back in.. Deleted DAG and recreated it… that didn’t work either

Tried using Exchange management shell but it didn’t work too

Checking the log coming in the description, I find out the log is complaining about not finding DAG while trying to resolve it.

Also the log says that it has installed Failover cluste rbut still the cluster can’t find FQDN called DAG.

So I will have to configure DAG in the dns and give it an IP of my first Exchange server

Checking DAG resolving from Exchange server

As the log says, restart is required after installing failover cluster so I’ll restart Exchange servers and then retry to add Exchange servers to the dag.

After restarting the server, It seems that things are working

The second server gave the following error

The Microsoft Exchange Replication service does not appear to be running on “EXCH2k1602”. Make sure that the server is operating, and that the services can be queried remotely.

Apparently the error is correct, After restarting the server it turned out that most of the second Exchange server’s services were not working..

To be honest I didn’t ask myself why did not the services started since I am using a preview version of Exchange 2016 and Windows as well so I manually started all the services.

Interestingly while checking services, I noticed new services e.g. (DAG Management, Compliance Audit, Notifications broker)

After starting the services, now I tried to add the second server again to the DAG.

So eventually, DAG doesn’t need an IP address but still a DNS value needs to be created for the NCO object and needs to have an IP assigned to it which will be the Exchange server IP address..

Next: I will add a database copy and see how it’s improved and do I need to restart the IS service as in Exchange 2013.

I will leave all the default values and add the second server for the database to be copied on. Unlike Exchange 2013 in most of the times the database would fail first and gives an error ..

In 2016 it starts directly seeding the database to the second Exchange server that’s member of the DAG.

On the second Server where the database has been copied to, I checked the Logs and Fastsearch was throwing errors as usual since database logs are not copied … as soon as the database logs finished copying the fastsearch will return ok and the database will appear as Healthy in EAC.

Fastsearch finally reported that indexing started on the newly copied DB.

The database copy should now report healthy in the EAC.

Hope you find this useful.
In the next article I will publish the Exchange server online to check the rest of the functionalities.

Exchange Migration, Exchange Online, Microsoft Exchange, Office 365

Export and import PST to Office 365 Exchange online mailboxes

August 6, 2015 Mohammed Hamada Leave a comment

Microsoft has launched a new feature that allows administrators to import PST to Exchange online directly through the portal.

In this article I’ll guide you through the steps of uploading one PST file and import it to a user’s mailbox. Although the steps are identical to Microsoft’s TechNet article but it’s more detailed and with screenshots. As I was personally reading and following the article at some point I got confused as to what which shared folder is the article speaking of and little tiny bits that are not detailed since Microsoft combines two methods in the same page “Ship data on Physical Drives and Upload files over the network”

So to achieve this, you’ll have to first sign in to your Office 365 portal. Open Exchange admin center and follow the below steps:

Granting Permission

Grant your self-importing PST permission to users by navigating to Exchange admin center -> Permissions> Double click on Compliance Management

Under Roles: click on + and add Mailbox Import Export role

Click on + Under Members and add your user account

2. `Copy Secure URL and secure storage account key

To get the Azure secure storage account key and URL you will have to go back to the Office 365 portal and then click on Import tab on the left pane

Then click on the Key sign below

When you click on it, you will be able to retrieve the key and the URL by clicking on Copy Key and URL .

The copy is pretty long and you’ll have to notice that sometimes you might get confused and copy only the appearing portion of it in the field… if you do so and copied that in the Azcopy command or Azure storage explorer you might get an error …

Here’s my Secure Storage account key that I am using on a trial version of Office 365.

KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g==

The URL has an important part which you will be using in Azure Storage Explorer tool in order to login and browse your Tenant’s storage which you’ll use to upload PST to.

The URL will appear as following.. You will need to copy the part highlighted in bold

https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/

You have to copy this in red in to the storage account name

d49d7ae0e38a4d8e9c93565

3. Copying PST files to Azure Folder using Azcopy command or Azure Storage Explorer (You can use Azure Storage Explorer too)

In order to upload PST files to Azure, you have two methods. The first is using Azcopy command which is pretty easy and straightforward (but still CMD dependent) or you can use the GUI Application which is Azure Storage explorer

To download azcopy, you can use the following link

http://az635501.vo.msecnd.net/azcopy-3-2-0/MicrosoftAzureStorageTools.msi

Or download them from the Import page as well under Resources:

Once the tool is installed. Right click on it and open it as administrator

The following command will take all the files inside my local folder path C:UsersMohammedDesktopupload

It will create a folder in Azure’s default folder ingestiondata called “Server01/PSTshareR1/”

It will use the destkey that I have retrieved from Office 365 Import window. And will leave all the logs in your local drive c:PSTuploadUploadlog.log

AzCopy /Source:C:UsersMohammedDesktopupload /Dest:https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/SERVER01/PSTshareR1/

/Destkey:KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g== /S /V:C:PSTUploadUploadlog.log

To make sure that files are uploaded. I will open Azure Storage Explorer 6 (Preview) and click Add Account on top

On add storage account window I will use the blob name that I have got from the URL earlier and storage secure key in the storage account key below and click on save.

Once I click that I will get a list of directories .. The default directory which is used by Office 365 is the “Ingestiondata” folder .. There our files will be uploaded.

https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

4. Create CSV File to import PST

Assuming you have 150 PST files that you want to upload and import into users which already have been enabled on Exchange online … In order to do so you will have to prepare a CSV file that looks like the below sample

To provide an explanation of what each column stands for .. Microsoft has written a table that clears the dust but some parts were not even clear for me like the FilePath as in the TechNet article it gets you confused with the “Ship data on Physical hard drives” since it uses your drive to upload data directly to Azure through the Import tool on Office 365 portal.

From <https://technet.microsoft.com/library/ms.o365.cc.IngestionHelp.aspx?v=15.1.166.0&l=1&f=255&MSPPError=-2147217396>

Note:

The friendly path here is the path of the folder you have created in Azure through the Azcopy command

AzCopy /Source:C:UsersMohammedDesktopupload /Dest:https://d49d7ae0e38a4d8e9c93565.blob.core.windows.net/ingestiondata/SERVER01/PSTshareR1/

/Destkey:KA9Z00rEYa1JlqGE4wO222MnsN5ywT0elOgLeNht/fSMIJPe2134hEChuuDJ5mfdknq8ts0+cez6uUvFzcQd6g== /S /V:C:PSTUploadUploadlog.log

So the CSV File is ready.

In Azure Storage Explorer I doubled check if the PST files has finished uploading and it’s there.

5. Using the Upload Files over the network

Back to Office 365 portal, go to Import and click on the + Sign and select Upload files over the network

Select I have access to the mapping file as well

Click on + and upload the CSV file that you have prepared for the mapping

Next File is imported, Click on “By checking this box, you agree to the terms and conditions of this service.

As soon as you accept and click next the Import is going to check path, email, folder and will start the import process.

Email before importing

Imported started, folder has been created

Importing is done

‘

Importing is done

Reference

https://technet.microsoft.com/library/ms.o365.cc.IngestionHelp.aspx?v=15.1.166.0&l=1&f=255&MSPPError=-2147217396#BKMK_CreateAnewMappingtoupload

https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

Exchange 2013, Microsoft Exchange, Office 365, Office365

Exchange 2013 ECP (Admin Panel Page) Fails with 500 Unexpected error after running Hybrid Configuration Wizard with Office 365

August 3, 2015 Mohammed Hamada Leave a comment

Symptoms

After you attempt to try Hybrid Configuration Wizard between Exchange 2013 SP1 and Exchange online (Office 365) .. You are unable to login to your OWA/ECP Page and instead you get an error:

Once you look at your event viewer you notice it records the event ID 4 with the following error

Cause:

The HCW or “Hybrid Configuration Wizard” In Exchange 2013 (CU6 or 8) might cause some changes to your CAS folder in the following path and file

“c:Program FilesMicrosoftExchange ServerV15ClientAccessecpDDI”` the file name is RemoteDomains.xaml and multiply some variables which causes ECP to fail and report that error.

Error:

Current user: ‘Domain.local/User’

Request for URL ‘https://ex2k1301.Domain.local:444/ecp/default.aspx(https://mail.Domain.com/ecp/)‘ failed with the following error:

System.Web.HttpUnhandledException (0x80004005): Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Data.DuplicateNameException: A column named ‘TargetDeliveryDomain’ already belongs to this DataTable.

at System.Data.DataColumnCollection.RegisterColumnName(String name, DataColumn column)

at System.Data.DataColumnCollection.BaseAdd(DataColumn column)

at System.Data.DataColumnCollection.AddAt(Int32 index, DataColumn column)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase.CreateColumn(DataTable table, Dictionary`2 rbacMetaData)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(Service profileBuilder)

at Microsoft.Exchange.Management.DDIService.AutomatedDataHandlerBase..ctor(String schemaFilesInstallPath, String schema)

at Microsoft.Exchange.Management.DDIService.WSListDataHandler..ctor(String schemaFilesInstallPath, String resourceName, String workflowName, DDIParameters parameters, SortOptions sortOptions)

at Microsoft.Exchange.Management.DDIService.DDIServiceHelper.GetListCommon(DDIParameters filter, SortOptions sort, Boolean forGetProgress)

— End of inner exception stack trace —

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

at Microsoft.Exchange.Management.ControlPanel.WebServiceReference.GetList(DDIParameters filter, SortOptions sort)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.LoadTargetDeliveryDomain(AddValueHandler addValue, LogErrorHandler logError)

at Microsoft.Exchange.Management.ControlPanel.OrganizationCache.TryGetValue[T](String key, T& value)

at Microsoft.Exchange.Management.ControlPanel._Default.RenderMetroTopNav()

at ASP.default_aspx.__RendermainForm(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)

at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at ASP.default_aspx.__Render__control1(HtmlTextWriter __w, Control parameterContainer)

at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)

at System.Web.UI.Page.Render(HtmlTextWriter writer)

at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

at System.Web.UI.Page.HandleError(Exception e)

at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

at System.Web.UI.Page.ProcessRequest()

at System.Web.UI.Page.ProcessRequest(HttpContext context)

at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.Data.DuplicateNameException: A column named ‘TargetDeliveryDomain’ already belongs to this DataTable.