Microsoft’s High Availability solutions

A high-availability solution masks the effects of a hardware or software failure and maintains the availability of applications so that the perceived downtime for users is minimized. The basics of high availability are already discussed in an earlier blog. This blog focuses on the various high availability solutions provided by Microsoft. The high availability solutions provided by Microsoft are as follows:

Failover Cluster

A failover cluster is a group of independent computers that work together to increase the availability of applications and services. The clustered servers (called nodes) are connected by physical cables and by software. All nodes in this group are managed as a single system and share a common namespace. If one of the cluster nodes fails, another node begins to provide service (a process known as failover). With this, users experience a minimum of disruptions in service.

More details: http://www.availabilitydigest.com/public_articles/0504/microsoft_cluster.pdf

Database mirroring

Database mirroring maintains two copies of a single database that must reside on different server instances of SQL Server Database Engine. Typically, these server instances reside on computers in different locations. One server instance serves the database to clients (the principal server). The other instance acts as a hot or warm standby server (the mirror server), depending on the configuration and state of the mirroring session. When a database mirroring session is synchronized, database mirroring provides a hot standby server that supports rapid failover without a loss of data from committed transactions. When the session is not synchronized, the mirror server is typically available as a warm standby server (with possible data loss). Database mirroring is implemented on a per-database basis and works only with databases that use the full recovery model.

More details: http://msdn.microsoft.com/en-us/library/ms189852.aspx

Log Shipping

Log shipping mechanism is based on replicating the changes made in primary database to the secondary (backup) databases by refereeing the transaction logs generated from the primary database. Log Shipping configuration automatically send transaction log backups from a primary database on a primary server instance to one or more secondary databases on separate secondary server instances. The transaction log backups are applied to each of the secondary databases individually. An optional third server instance, known as the monitor server, records the history and status of backup and restores operations and, optionally, raises alerts if these operations fail to occur as scheduled.

More details: http://msdn.microsoft.com/en-us/library/ms187103.aspx

Data Replication:

Data Replication is a set of technologies for copying and distributing data and database objects from one database to another and then synchronizing between databases to maintain consistency. The result is a distributed database in which users can access data relevant to their tasks without interfering with the work of others. Using replication, one can distribute data to various locations and to remote or mobile users over local and wide area networks, dial-up connections, wireless connections, and the Internet.

Database replication can be done in at least three different ways:

• Snapshot replication: Data on one server is simply copied to another server, or to another database on the same server.
• Merging replication: Data from two or more databases is combined into a single database.
• Transactional replication: Users receive full initial copies of the database and then receive periodic updates as data changes.

More details: http://msdn.microsoft.com/en-us/library/ms151198.aspx

AlwaysOn:

AlwaysOn, a new high availability solution available with SQL Server 2012, is a combined form of the Microsoft’s existing High-Availability (HA) and Disaster Recovery (DR) functionalities like database mirroring, failover-clustering and log shipping. This integration makes them work better together and removes a lot of customer-required set up and tuning, thus helping eliminate potential errors. An AlwaysOn solution can leverage two major SQL Server 2012 features for configuring availability at both the database and the instance level:

• AlwaysOn Availability Groups, new in SQL Server 2012, greatly enhance the capabilities of database mirroring and helps ensure availability of application databases, and they enable zero data loss through log-based data movement for data protection without shared disks. Availability groups provide an integrated set of options including automatic and manual failover of a logical group of databases, support for up to four secondary replicas, fast application failover, and automatic page repair.
• AlwaysOn Failover Cluster Instances (FCIs) enhance the SQL Server failover clustering feature and support multisite clustering across subnets, which enables cross-data-center failover of SQL Server instances. Faster and more predictable instance failover is another key benefit that enables faster application recovery.

More details: http://msdn.microsoft.com/en-us/library/hh781257.aspx

There is some basic difference between the scenarios for implementation for each of these technologies, which will be covered in a further blog.

References:

• High Availability Solutions Overview (http://msdn.microsoft.com/en-us/library/ms190202.aspx)
• Failover Clusters (http://technet.microsoft.com/en-us/library/cc732488(v=ws.10).aspx)
• Database Mirroring Overview (http://msdn.microsoft.com/en-us/library/ms189852.aspx)
• Log Shipping Overview (http://msdn.microsoft.com/en-us/library/ms187103.aspx)
• SQL Server Replication (http://msdn.microsoft.com/en-us/library/ms151198.aspx)
• AlwaysOn (http://msdn.microsoft.com/en-us/library/hh781257.aspx)

SAP selects IBM DB2 as a strategic database platform

Very recently, SAP AG adopted IBM DB2 as the database of choice for reducing the complexity and operational costs of its IT landscape, particularly Human Capital Management (HCM), ERP and Business Intelligence applications. The case study outlines how SAP saw major benefits, including:

- Improved response times and efficiency

- High user productivity

Summary:

The Challenge

SAP IT (the IT department of SAP AG) wanted to be able to take advantage of new SAP software functionalities while reducing the complexity and operational costs of its IT landscape. The company also wanted to move to a new database platform to deliver optimal performance.

The Solution

In three separate projects, SAP IT upgraded its Human Capital Management (HCM), ERP and Business Intelligence applications, simultaneously performing Unicode conversion and migrating databases from Oracle to IBM DB2.

Quotes:

“The efficiency of the new IBM DB2 solution has given us headroom within our database and storage servers to grow as business workload rises, with high user productivity and great return on investment.”

~Peter Boegler, Solution Architect, SAP IT

Another quotable quote:

“In addition to the Unicode conversion, we also had a corporate objective to migrate away from our existing Oracle database and onto IBM DB2, which is now the recommended database for SAP software,” says Peter Boegler.

(Real time Case Study)

SAP HANA is real real-time technology

Why is SAP HANA so fast?

In recent time, SAP released its incredible innovative idea of In-Memory Computing for providing real-real time computing, along with an appliance SAP HANA (High Performance Analytical Appliance). It is claimed to be a game changing innovation, which has forced other vendors to take a step forward and jump into this dimension of technology.

As claimed by SAP, the HANA appliance achieved outstanding statistical figures:

• Ability to scan 2 million records per millisecond per core and over 10 million complex aggregations calculated on the fly per second per core.
• Full parallelization at 1,000 cores and beyond
• 450 billion record system implemented on less than three terabytes of physical memory.

An explainination of this outstanding performance, as given by John Appleby, Head of Business Analytics & Technology, Bluefin Solutions stands as:

"Regular RDBMS technologies put the information on spinning plates of iron (hard disks) from which the information is retrieved. HANA stores information in electronic memory, which is some 50x faster (depending on how you calculate). HANA stores a copy on magnetic disk, in case of power failure or the like. In addition, most SAP systems have the database on one system and a calculation engine on another, and they pass information between them. With HANA, this all happens within the same machine."

(content taken from publicly available information available here)

This really shows a good potential of a game changing technology breakthrough, forcing vendors to work on the In-memory concept. And if this continues in the same fashion, one day, everyone will be having a super computer (as we know them today) into the pockets of common people.

A quick brief about SAP

SAP is one of the world’s largest vendors of independent enterprise software, that provides specialized software applications for enterprise resource planning (ERP), customer relationship management (CRM), supply chain management (SCM), product lifecycle management (PLM), software supplier relationship management (SRM), and business intelligence (BI).

SAP provides enterprise software solutions (like SAP R/3) to a large number of multinational companies who require fast, stable and reliable business applications. These applications (categorized as platform applications, extension applications and composite applications) enable them to handle millions of transactions in multiple currencies and possess international presence. These applications are available in form of smaller components, which can be customized and configured as per the needs of the industry.

In case there is any need of further modification or functionality enhancement, these applications can be customized using the SAP Netweaver development technologies. SAP Netweaver development technologies includes ABAP (programming language along with an integrated development environment, something similar to COBOL), Java, and Composition development framework (tools and environment across heterogeneous systems).

High Availability Basic Concepts-I

For any software application or service, high availability refers to the availability of that application or service to its users without any failure. For simplest example, Google is providing its search capabilities to all the Internet users (virtually) for 24 X 7 via its Google search engine. We assume that as soon as we switch on our PC or laptop (or any other compatible device), and connect to Internet, Google search will be available to us. The use of word virtually compensates for those small time periods, when Google search engine is not available to the users due to server maintenance, or some other reasons. This duration will be called as downtime, and is usually measured over a year. So if any application of service provider claims 99.9% availability, it means that over a year’s time, its services may be down for 0.1% duration of year, i.e. 8 hours and 45 minutes.

The primary goal of any high availability solution is to minimize the impact of downtime. And the Service Level Agreements for any such high availability solution (or service) always covers these clarifications in its terms and conditions. The availability of a solution (application or service, or a group of them) can be expressed as this calculation

Availability = ( Actual Uptime / Expected Uptime ) x 100:

The resulting value is often expressed by industry in terms of the number of 9’s that the solution provides; meant to convey an annual number of minutes of possible uptime, or conversely, minutes of downtime.

Number of 9’s	Availability Percentage	Total Annual Downtime
2	99%	3 days, 15 hours
3	99.9%	8 hours, 45 minutes
4	99.99%	52 minutes, 34 seconds
5	99.999%	5 minutes, 15 seconds

More details on various high availability applications and services will be covered very soon. (as soon as I'll have free time available for the same :)

Alright, got some time to generate some content as a blog Microsoft's High Availability Solutions.

Journey of Crystal Reports

For one of the recent assignments, I had to dig out the entire history of the world famous reporting product “Crystal Reports”, as it originated, transformed, and evolved into what we see as SAP Crystal Reports 2011 today. Here is the brief summary of the same:

Crystal Reports is a business intelligence application used to design and generate reports from a wide range of data sources. Started as Crystal Services Inc. in 1989, the company developed this product as a commercial report writing tool for their accounting software. They released three initial versions of the product as Quik Reports (1990), Quik Reports 2.0 (1991) and Quik Reports 3.0 (1992).

Then, after acquisition by Seagate Technology Inc in 1994, the company was named as Seagate Software. The product was also renamed and launched as Seagate Info 4.0 in 1995. In 1995, Seagate Software decided to have collaboration with Holistic Systems (acquired by Seagate technologies Inc.) forming Information Management Group of Seagate Software. Under this collaboration, the product was rebranded, and users enjoyed 5 versions, namely Crystal Reports 4.5 (1996), Crystal Info 5 (1997), Seagate Crystal Info 6 (1998), Seagate Info 7 (1999), and Seagate Info 7.5 (2000). (And yes, the name was changed almost after every release; they were just not able to find the right name for it!!).

In year 2001, the company Seagate Software was again renamed as Crystal Decisions. It then released the versions Crystal Enterprises 8.0, Crystal Enterprises 8.5 (2001) and then Crystal Enterprises 9.0 (2001) in quick successions.

Then it was again acquired by famous Business Intelligence solution provider BusinessObjects in 2003. The first version released after this acquisition carried the same naming format as its earlier trend, Crystal Enterprise 10.0 in year 2004. Then the product was released with revised names as Crystal Reports XI R1 (2005), and Crystal Reports XI R2 (2006).

With acquisition of BusinessObjects by SAP in 2007, this product again witnessed change of name. It was released as Crystal Report 2008 (2008).

The latest release is named as SAP Crystal Report 2011.

For details on future roadmap of this product, and what users can expect from SAP for this product, here is the SAP Crystal Reports 2011 and the 20-year roadmap.

Business Intelligence Technology Stack

Business Intelligence general refers to identification, extraction or transformation of business data into useful information (reports, charts, graphs etc.) to gain business specific insights like demand forecasts and sales predictions, thus providing better decision making capabilities. It usually refers to the computer based techniques, like reporting, analytics, data mining, benchmarking, predictive analysis etc., but is not limited to them.

As explained by D. J. Power in his work “A Brief History of Decision Support Systems”, there are various tools and technologies that provide Business Intelligence capabilities, and providing an efficient Decision Support System (DSS). His research covers even the basic systems like file drawers, which are used to keep information in organized and readily searchable manner (for small organizations). But in present information age, those kinds of systems seems outdated for requirements of a global organizations, with hundreds of branches across the world, and

generating and processing huge amount of information per hour. In this article, we are focusing only on computer based programs and applications, that consumes and processes the digital information available on organization’s servers, and then generates meaningful results out of it, which provokes better decisions from BDMs (Business Decision Makers), TDMs (Technology Decision Makers) or other IT Pros involved in decision making.

The well-known enterprise analyst organization Gartner predicts a five-fold growth in the Open-Source BI tools product deployment by the end of 2012. They also mentioned in their report on Magic Quadrant for Business Intelligence Platforms 2011, that the growth in BI will be driven by factors like Consumerization of BI and support for extreme data performance with emerging data sources (known as Big Data). And with some recent break-through innovations by the major BI vendors (like SAP’s HANA appliance, Oracle’s Exalytics appliances,

and Microsoft’s BISM model), IT world may expect more surprises coming from the major BI vendors (including but not limited to Microsoft, Oracle, microstrategy, IBM, Information Builders, QlikTech, SAP and SAS).

But irrespective of vendor, all BI solutions have a generic technology stack, with following layers:

· User Interface: This includes the Web based or application based frontend that brings the analysis to the users. It includes the portals (in case of networked or web-based analytics) or the application front end in case of locally deployed BI solution.

· Development and Admin Tools: This comprises of the tools, languages and processes involved in the development and management of BI applications and systems. The difference between BI systems and BI solutions will be covered in another blog. For example, some BI development languages can be MultiDimensional eXpressions (MDX), XML for Analysis (XMLA), Data Mining Extensions (DMX) etc.

· BI Tools: This comprises of the tools (reports, dashboards or otherwise) that enables the users to perform the desired analysis on the underlying data. User access these tools via the User Interface layer discussed above. For instance, Microsoft’s PowerPivot and Power View, SAP’s crystal reports, Jaspersoft, Oracle’s Business Intelligence Foundation Suite are just few examples to name, there are more than 100 of readily usable vendor products available in the market.

· Applications and BI data sources: This comprises of the various sources that keep the information in pre-processed form that can be readily consumed for analysis. This includes models like Online Analytical Processing (OLAP) Cubes or Decision Support Systems, and concepts like Data Mining, Analysis Services, etc.

· Data Integration Tools: This comprises of the various data management tools and concepts like Master Data Management (covering data collection, source identification, schema mapping, normalization, data transformation, rule administration, error detection and correction, data consolidation, data storage, data distribution, data classification, item master creation, data enrichment and data governance) and services like taxonomy services, Data Quality Services,

· Data warehouse platform: This comprises of various data sources, including simple text based files, excel sheets, relational databases, or even complex unstructured data types like audio files, videos, web-logs, click-streams and geo-spatial data etc.