Alwayson Availability Group Replica is Not Synchronizing (Error: 976, Severity: 14, State: 1.)

For some reason you can lose communication between the primary and secondary replicas.

As known, there is a policy on SQL Server to check the data synchronization state of the database replica. The policy is in an unhealthy state when the data synchronization state is NOT SYNCHRONIZING or the state is not SYNCHRONIZED for the synchronous-commit database replica.

After investigating the sql server errorlog, I got the following msg:

The target database, ‘YourDatabase’, is participating in an availability group and is currently not accessible for queries. Either data movement is suspended or the availability replica is not enabled for read access. To allow read-only access to this and other databases in the availability group, enable read access to one or more secondary availability replicas in the group.  For more information, see the ALTER AVAILABILITY GROUP statement in SQL Server Books Online. (Microsoft SQL Server, Error: 976)

According to Technet, this issue can be caused by the following:

  • The availability replica might be disconnected.
  • The data movement might be suspended.
  • The database might not be accessible.
  • There might be a temporary delay issue due to network latency or the load on the primary or secondary replica.

Further information:

You can fix it, using the following T-SQL command to force the resume synchronization:



You can also follow the link: How to resume an AG database.

Check out these related tips:

How to monitor Timeout and Changes Roles from Availability Groups with Alerts
How to Monitor AlwaysON – Primary / Secondary Replicas


Default trace security auditing [SQL Server]

The Default trace enabled can provides us valuable information about the running server, but it isn’t well-documented.

What is the default trace? The default trace is enabled by default in SQL Server and is a minimum weight trace which consists by default of five trace files ( .trc) located in the SQL Server installation directory. The files are rolled over as time passes.

The script below will give you the most recently manipulated objects in your databases.

SELECT  TE. name ,
        v .subclass_name ,
        DB_NAME(t .DatabaseId) AS DBName ,
        T .NTDomainName ,
        t .NTUserName ,
        t .HostName ,
        t .ApplicationName ,
        t .LoginName ,
        t .Duration ,
        t .StartTime ,
        t .ObjectName ,
        CASE t. ObjectType
          WHEN 8259 THEN 'Check Constraint'
          WHEN 8260 THEN 'Default (constraint or standalone)'
          WHEN 8262 THEN 'Foreign-key Constraint'
          WHEN 8272 THEN 'Stored Procedure'
          WHEN 8274 THEN 'Rule'
          WHEN 8275 THEN 'System Table'
          WHEN 8276 THEN 'Trigger on Server'
          WHEN 8277 THEN '(User-defined) Table'
          WHEN 8278 THEN 'View'
          WHEN 8280 THEN 'Extended Stored Procedure'
          WHEN 16724 THEN 'CLR Trigger'
          WHEN 16964 THEN 'Database'
          WHEN 16975 THEN 'Object'
          WHEN 17222 THEN 'FullText Catalog'
          WHEN 17232 THEN 'CLR Stored Procedure'
          WHEN 17235 THEN 'Schema'
          WHEN 17475 THEN 'Credential'
          WHEN 17491 THEN 'DDL Event'
          WHEN 17741 THEN 'Management Event'
          WHEN 17747 THEN 'Security Event'
          WHEN 17749 THEN 'User Event'
          WHEN 17985 THEN 'CLR Aggregate Function'
          WHEN 17993 THEN 'Inline Table-valued SQL Function'
          WHEN 18000 THEN 'Partition Function'
          WHEN 18002 THEN 'Replication Filter Procedure'
          WHEN 18004 THEN 'Table-valued SQL Function'
          WHEN 18259 THEN 'Server Role'
          WHEN 18263 THEN 'Microsoft Windows Group'
          WHEN 19265 THEN 'Asymmetric Key'
          WHEN 19277 THEN 'Master Key'
          WHEN 19280 THEN 'Primary Key'
          WHEN 19283 THEN 'ObfusKey'
          WHEN 19521 THEN 'Asymmetric Key Login'
          WHEN 19523 THEN 'Certificate Login'
          WHEN 19538 THEN 'Role'
          WHEN 19539 THEN 'SQL Login'
          WHEN 19543 THEN 'Windows Login'
          WHEN 20034 THEN 'Remote Service Binding'
          WHEN 20036 THEN 'Event Notification on Database'
          WHEN 20037 THEN 'Event Notification'
          WHEN 20038 THEN 'Scalar SQL Function'
          WHEN 20047 THEN 'Event Notification on Object'
          WHEN 20051 THEN 'Synonym'
          WHEN 20549 THEN 'End Point'
          WHEN 20801 THEN 'Adhoc Queries which may be cached'
          WHEN 20816 THEN 'Prepared Queries which may be cached'
          WHEN 20819 THEN 'Service Broker Service Queue'
          WHEN 20821 THEN 'Unique Constraint'
          WHEN 21057 THEN 'Application Role'
          WHEN 21059 THEN 'Certificate'
          WHEN 21075 THEN 'Server'
          WHEN 21076 THEN 'Transact-SQL Trigger'
          WHEN 21313 THEN 'Assembly'
          WHEN 21318 THEN 'CLR Scalar Function'
          WHEN 21321 THEN 'Inline scalar SQL Function'
          WHEN 21328 THEN 'Partition Scheme'
          WHEN 21333 THEN 'User'
          WHEN 21571 THEN 'Service Broker Service Contract'
          WHEN 21572 THEN 'Trigger on Database'
          WHEN 21574 THEN 'CLR Table-valued Function'
          WHEN 21577
          THEN 'Internal Table (For example, XML Node Table, Queue Table.)'
          WHEN 21581 THEN 'Service Broker Message Type'
          WHEN 21586 THEN 'Service Broker Route'
          WHEN 21587 THEN 'Statistics'
          WHEN 21825 THEN 'User'
          WHEN 21827 THEN 'User'
          WHEN 21831 THEN 'User'
          WHEN 21843 THEN 'User'
          WHEN 21847 THEN 'User'
          WHEN 22099 THEN 'Service Broker Service'
          WHEN 22601 THEN 'Index'
          WHEN 22604 THEN 'Certificate Login'
          WHEN 22611 THEN 'XMLSchema'
          WHEN 22868 THEN 'Type'
          ELSE 'Hmmm???'
        END AS ObjectType
FROM    [fn_trace_gettable]( CONVERT(VARCHAR (150), ( SELECT TOP 1
                                                    FROM    [fn_trace_getinfo](NULL)
                                                    WHERE   [property] = 2
                                                  )), DEFAULT ) T
        JOIN sys .trace_events TE ON T. EventClass = TE .trace_event_id
        JOIN sys .trace_subclass_values v ON v. trace_event_id = TE .trace_event_id
                                            AND v. subclass_value = t .EventSubClass
WHERE IN ( 'Object:Created', 'Object:Deleted', 'Object:Altered' )AND
                -- filter statistics created by SQL server                                        
        --t.ObjectType NOT IN ( 21587 )-- filter tempdb objects
         DatabaseID <> 2
              -- AND NTUserName <> 'username'
              -- get only events in the past 24 hours
        AND StartTime > DATEADD(HH, -24, GETDATE())
ORDER BY t.StartTime DESC ;

If you need to enable:

sp_configure 'show advanced options', 1;
sp_configure 'default trace enabled', 1;

I hope this was useful.

NodeRunner process in SharePoint 2013

The node runner application used by SharePoint 2013 preview for search related process slows down the system due to memory leak issues.

To fix this issue follow the below 2 steps

1. Open SharePoint PowerShell window and execute the below command

Set-SPEnterpriseSearchService -PerformanceLevel Reduced

2. Open the config file at C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0\noderunner.exe.config and change the value of memoryLimitMegabytes in node to some other value other than 0, say 100 or 200 which sets the cap limit for memory to be used.

3. Restart the machine

Introducing Cloud Witness – [SQL Server]

We know that when we create a Cluster environment between two sites, we must take care about Quorum Witness.
If you need automatic failover SLA in your company, the best way is to separate the (Quorum) FileShare Quitness at a third site. So it is not lost if one of the other two sites has problems.

But, What if you dont have the third site?

Now, It’s possible using Cloud Witness. A new type of Failover Cluster quorum witness with Windows Server 2012 R2.
Follow the link below to an article with steps required to configure it.

Error while installing the Pre-requisite for SharePoint Server 2013

When I was trying to install the SharePoint 2013 pre requisite on Windows Server 2008 R2 Std SP1, I got an error message:

2015-02-02 11:14:27 – “C:\Windows\system32\cscript.exe” “C:\Windows\system32\iisext.vbs” /enext “ASP.NET v4.0.30319”
2015-02-02 11:14:29 – Install process returned (-2146646015)
2015-02-02 11:14:29 – [In HRESULT format] (-2146646015)
2015-02-02 11:14:29 – Error when enabling ASP.NET v4.0.30319
2015-02-02 11:14:29 – Last return code (-2146646015)

Start the CMD.exe as Administrator

Execute this command: “C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe” -i -enable

Alert – Databases do not have Transaction log backups

The SP below sends alert when there are no transaction log backups in the last 90 minutes.

CREATE PROCEDURE [dbo].[usp_NoTlogBackup] (@Length INT=90) -- 90 Minutes
DECLARE @BadDatabases VARCHAR(8000)

SELECT @BadDatabases = STUFF(
(select ', ' + cast(a.database_name as varchar(100))
 select [name] as database_name
 from master.dbo.sysdatabases
 where databasepropertyex([name], 'recovery') in ('FULL','BULK_LOGGED') 
 and databasepropertyex([name], 'isinstandby') = 0
 and databasepropertyex([name], 'status') = 'online'
 and databasepropertyex([name], 'updateability') = 'read_write'
 and [name] not in ('model','tempdb')
 ) a
 left join msdb.dbo.backupset b
 on a.database_name = b.database_name 
 and b.type='L' 
 and datediff(hour,b.backup_finish_date,getdate()) < @Length
 where b.database_name is null

IF (@BadDatabases IS NOT NULL)

declare @ServerT varchar(100)
select @ServerT = 'Alert Backup - '+@@SERVERNAME

SET @Body = 'On ' + @@SERVERNAME + ' The following Databases do not have Transaction log backups: ' + @BadDatabases

 EXEC msdb . dbo. sp_send_dbmail        
      @recipients=N';' ,
      @body = @Body ,         
      @subject = @ServerT,         
      @profile_name ='profile_name',         
      @body_format = 'HTML'


How to list all objects and indexes [SQL Server]

Today I had to list which objects belongs to which filegroup.
It’s a simple and useful script that you can take from system catalogues: sys.filegroups, sys.indexes, sys.database_files and sys.data_spaces.

SELECT Object_name(i.[object_id]) AS [ObjectName],
       i.[index_id]               AS [IndexID],
       i.[name]                   AS [IndexName],
       i.[type_desc]              AS [IndexType],
       f.[name]                   AS [FileGroup],
       d.[physical_name]          AS [DatabaseFileName]
FROM   [sys].[indexes] i
       LEFT JOIN sys.partition_schemes pf
              ON pf.[data_space_id] = i.[data_space_id]
       LEFT JOIN [sys].[filegroups] f
              ON f.[data_space_id] = i.[data_space_id]
       LEFT JOIN [sys].[database_files] d
              ON f.[data_space_id] = d.[data_space_id]
       LEFT JOIN [sys].[data_spaces] s
              ON f.[data_space_id] = s.[data_space_id]
WHERE  Objectproperty(i.[object_id], 'IsUserTable') = 1
ORDER  BY Object_name(i.[object_id]),

Take a look this sample


In the next post I’m going to share How Move Index to other FileGroup.