Archive for the ‘SQL’ Category.

Date Dimension Stored procedure using a CTE

Have been playing around a little more making my time dimension creation and population a little more efficient

Let’s say we have a dimension table created like so


CREATE TABLE [dbo].[CALENDAR](
[CAL_DATE] [datetime] NULL,
[CAL_DAY] [varchar](50) NULL,
[CAL_DAY_IN_WEEK] [int] NULL,
[CAL_DAY_IN_MONTH] [int] NULL,
[CAL_DAY_OF_YEAR] [int] NULL,
[CAL_WEEK] [int] NULL,
[CAL_MONTH] [varchar](50) NULL,
[CAL_SHORT_MONTH] [varchar](50) NULL,
[CAL_MONTH_IN_YEAR] [int] NULL,
[CAL_QUARTER] [int] NULL,
[CAL_HALF_YEAR] [int] NULL,
[CAL_YEAR] [int] NULL,
[IS_WEEKEND] [int] NULL,
[WORKING_DAY_NO] [int] NULL
) ON [PRIMARY]
GO

We can then use a CTE to generate a list of data for this table efficiently (well, more efficiently than a loop)

Example: (generates data for 1 Jan 2010 to 31 Dec 2012).


WITH mycte as
(
select cast('2010-01-01' as datetime) DateValue,
case when datepart(dw, cast('2010-01-01' as datetime) -1) < 6 
then 0 else 1 end IS_WEEKEND

union all
select DateValue + 1,
case when datepart(dw, DateValue) < 6 then 0 else 1 end IS_WEEKEND
from mycte
where DateValue + 1 < '2012-12-31'
)
select DateValue as CAL_DATE
, datename(dw, DateValue) CAL_DAY
, datepart(dw, DateValue-1) CAL_DAY_IN_WEEK
, datepart(dd, DateValue) CAL_DAY_IN_MONTH
, datepart(dy, DateValue) CAL_DAY_OF_YEAR
, datepart(ww, DateValue) CAL_WEEK
, datename(mm, DateValue) CAL_MONTH
, substring(datename(mm, DateValue), 1, 3) CAL_SHORT_MONTH
, datepart(mm, DateValue) CAL_MONTH_IN_YEAR
, datepart(qq, DateValue) CAL_QUARTER
, case when datepart(mm, DateValue) < 7 then 1 else 2 end CAL_HALF_YEAR
, datepart(yy, DateValue) CAL_YEAR
, IS_WEEKEND
, (SELECT sum(IS_WEEKEND) FROM mycte
WHERE
DateValue and datepart(yy, DateValue) = datepart(yy, m.DateValue)
and datename(mm, DateValue) = datename(mm, m.DateValue)
) AS WORKING_DAY_NO
FROM mycte m
OPTION (MAXRECURSION 0)

So, to make this more efficient, how about wrapping this up in a stored procedure so we can just pass the start and end dates we need.


-- ==================================================================================
-- Author: Will Riley (www.wills-blog.com | Twitter @troublewithdata)
-- Create date: 2012-03-27
-- Description: Populate a date dimension table using a CTE based stored procedure
--
-- Note: Use it like so:-
--
-- INSERT INTO [dbo].[CALENDAR]
-- execute MyDateGenerator '2010-01-01','2011-12-31'
--
-- You also need to create the calendar table
-- ==================================================================================

USE [TestDW]
GO

IF EXISTS (SELECT * FROM sysobjects WHERE id = object_id(N’[dbo].[MyDateGenerator]‘) AND OBJECTPROPERTY(id, N’IsProcedure’) = 1)
BEGIN
DROP PROCEDURE [dbo].[MyDateGenerator]
END

GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

CREATE PROCEDURE [dbo].[MyDateGenerator]
@FromDate datetime, @ToDate datetime

AS BEGIN

SET NOCOUNT ON;

WITH mycte as
(
select cast(@FromDate as datetime) DateValue,
case when datepart(dw, cast(@FromDate as datetime) -1) < 6 then 0 else 1 end IS_WEEKEND
union all
select DateValue + 1,
case when datepart(dw, DateValue) < 6 then 0 else 1 end IS_WEEKEND
from mycte
where DateValue + 1 < @ToDate
)
select DateValue as CAL_DATE
, datename(dw, DateValue) CAL_DAY
, datepart(dw, DateValue-1) CAL_DAY_IN_WEEK
, datepart(dd, DateValue) CAL_DAY_IN_MONTH
, datepart(dy, DateValue) CAL_DAY_OF_YEAR
, datepart(ww, DateValue) CAL_WEEK
, datename(mm, DateValue) CAL_MONTH
, substring(datename(mm, DateValue), 1, 3) CAL_SHORT_MONTH
, datepart(mm, DateValue) CAL_MONTH_IN_YEAR
, datepart(qq, DateValue) CAL_QUARTER
, case when datepart(mm, DateValue) < 7 then 1 else 2 end CAL_HALF_YEAR
, datepart(yy, DateValue) CAL_YEAR
, IS_WEEKEND
, (SELECT sum(IS_WEEKEND) FROM mycte
WHERE
DateValue and datepart(yy, DateValue) = datepart(yy, m.DateValue)
and datename(mm, DateValue) = datename(mm, m.DateValue)
) AS WORKING_DAY_NO

FROM mycte m

OPTION (MAXRECURSION 0)
END
GO

Now all we need to do is to run it like so:


INSERT INTO [dbo].[CALENDAR]
execute MyDateGenerator '2010-01-01','2011-12-31'

I could imagine this being useful within an incremental SSIS package to add a new “chunk” of data to your time dimension…

Microsoft BI on iPad

I know that Microsoft have promised that they will be playing catch up later this year in terms of Mobile BI, and for sure they need to. Offerings available right now from the likes of Cognos, QlikView, Business Objects and Microstrategy (current personal favourite) far outstrip what you can do with the current Microsoft stack. But what does work?

Well, from what I can see, anything Sliverlight based is out, so that rules out PowerView and Decomp Trees in PPS. It seems that most of the other things work though, so there’s much you can still do.

Having managed to blag a company iPad in my new role as Reporting & Analytics lead, I figured I’d hook it up to the MSFT 2008R2 demo server I built that currently hosts some of Logica’s Spark Centre demos. Having installed Junos Pulse, a VPN app that allows me to securely connect to my work network, I found that the Sharepoint “pretend Telco” site renders quite well on the iPad.

20120309-201022.jpg

Firstly, I checked out SSRS… There’s no right click on iPad so you need to hit the screen over any drop downs and wait a moment for the selections to pop up.

20120309-201407.jpg

Looks pretty good… Next was PPS – remember, no Silverlight and doesn’t look like drill downs are fully working, but still able to do things like selecting chart items, changing from chart to grid and most impressively, export to Excel and PowerPoint works just fine (providing you have an Office programme installed such as QuickOffice, Docs to Go etc). Click Export to Excel and you get a choice..

20120309-201706.jpg

And here’s the report in QuickOffice

20120309-201848.jpg

Last thing to try was Excel services. Here, the Open in Excel function does not work. Apparently, there is no fooling it in to accepting being opened in a cheap substitute ;-) but the charts look OK…

20120309-202058.jpg

So, not perfect, but not all despair, and I’m assured that there are lots of goodies to come later in the year once the SQL2012 launch is out of the way. Still, it will need to be good to match my current favourites… If you get the chance, have a look at the Microstrategy iPad app….

20120309-202713.jpg

And the nice app from RoamBI

20120309-202821.jpg

SQL Server MDS excel addin error

When you are trying to connect to MDS from Excel, if you get the following error

“The Master Data Manager Web Application is Older than the Add-in Version”

while using MDS excel addin, then one possible reason is that the connection for the MDS excel addin does not like the full address i.e.

http://severname/default.aspx

You just need to enter http://servername/  or http://servername/mds and it works fine!

The error message is not clear for this issue!!

Download details: SQL Server 2008 R2 Update for Developers Training Kit (January 2011 Update)

This is a pretty good resource for those people wanting to get started with the latest versions of the Microsoft BI stack.

Download details: SQL Server 2008 R2 Update for Developers Training Kit (January 2011 Update)

Gartner Magic Quadrant for Business Intelligence Platforms

Using TFS 2010 with BIDS 2008 R2

Sadly, out of the box, you can’t connect from BIDS 2008 (the SQL 2008 or 2008 R2 version) to TFS 2010. I guess this will come with Denali where hopefully the new Juneau Development Environment will be integrated with TFS 2010.

I managed to get SQL Server 2008 R2 BI development tools (BIDS 2008) to work with TFS 2010 by doing the following:

 1. Install BI Development Tools that come with SQL Server 2008 R2

2. Install VS2008 Team Explorer (for BIDS/VS2008 from MSFT available here)

3. Install VS2008 Service Pack 1 (available here)

4. Install  VS2008 SP1 Forward Compatibility Update for TFS 2010 (available here)

Once I had done that, I was able to connect to a TFS 2010 Project and successfully check VS2008 projects in and out of source control, raise bugs etc.

Have attached a few screen grabs showing the integration from BIDS and also from the Web Client Portal (basically Sharepoint for TFS) showing some of the out of the box dashboard stuff.

Note that when entering a TFS server to connect to in BIDS, a change from TFS 2008 means that you now need to fully qualify your TFS Project

When asked for which TFS Server to connect to, type http://VM2008BITFS:8080/TFS (assuming you use default ports etc)

TFS Team Web Access

Team Web Access in TFS 2010

TFS Project Dashboard

Portal Dashboard in TFS 2010

BIDS Check In

BIDS check-in & source control access (similar to TFS 2008)

SSAS Maestro Programme

image

I received email confirmation today of my registration on the three-day deep-dive course on Analysis Services 2008 R2 and subsequent possibility of joining the SSAS Maestro Program. This is exciting as it’s going to be a real challenge to pass!

 

The new SSAS Maestro Program a three-day, deep-dive course on Analysis Services 2008 R2 gives architects and consultants the education and hands-on experience needed to deliver highest scalable OLAP solutions. Prepared and presented by top industry experts and the SQL Server Analysis Server team, this intensive course gives top SSAS professionals the education and hands-on experience needed to deliver highly complex and highly scalable OLAP solutions using Analysis Services 2008 R2. Today, there is increasing need for expertise to architect deliver and maintain mission critical OLAP deployments and the primary objective of this course is equip you with the latest SSAS best practices and case studies.

 

Agenda

Module 1: Analysis Services 2008 R2 and Large Cubes

· Overview and purpose of course

· Summary of modules

· Description of how course participation is evaluated


Module 2
: Tools for Monitoring and Optimizing Analysis Services

· Introducing Analysis Services Monitoring

· Executing Queries and Operations with Ascmd

· Monitoring Queries with SQL Server Profiler

· Monitoring the Server with Performance Monitor

· Using the Resource Monitor Project

· Using BIDS Helper

· Using Other Monitoring Tools


Module 3
: Internal Data Storage Structures

· Understanding Internal Data Storage Components

· Examining Data Structures for Attributes

· Examining Data Structures for Hierarchies

· Examining Data Structures for Partitions

· Optimizing Hardware for Random I/O


Module 4
: Memory Management

· Switching from the Analysis Server Heap to the Windows Heap

· Tuning the Economical Model of Memory Management

· Running Analysis Services With Other Software

· Tuning Memory Options for Processing

· Preallocating Memory to the Analysis Server


Module 5
: Thread Management

· Understanding the Trade-Offs in Thread Management

· Examining the Thread Pools

· Configuring the Number of Available Threads

· Handling Long-Running Queries

· Monitoring Thread Usage


Module 6
: Designing Partitions

· Understanding Partitions

· Sizing Partitions

· Defining the Partition Slice

· Partitioning for Query Performance

· Partitioning for Processing Performance

· Partitioning for Discount Count Measures

· Using Solid State Drives for Discount Count Partitions


Module 7
: Designing Aggregations

· Understanding Aggregations

· Considering Aggregations in Large Cubes

· Designing Aggregations

· Deciding for Each Attribute

· Tools and Tips


Module 8:
High Performance ROLAP Partitions for Large Cubes

· Choosing ROLAP Instead of MOLAP

· Simplifying the Cube Structure

· Working With Aggregations

· Developing a Partitioning Strategy

· Examining the ROLAP Benchmarks

· Examining Processor Utilization

· Handling Skewed Data Distributions

· Reviewing Lessons Learned and Best Practices


Module 9
: Large Cube Processing Strategies

· Understanding the Steps of Dimension Processing

· Choosing a Dimension Processing Option

· Examining Processing of Special Dimension Types

· Examining Processing and Storage Modes

· Understanding the Steps of Cube Processing

· Choosing a Cube Processing Option

· Optimizing Processing


Module 10
: Advanced Dimension Design Best Practices

· Working With:

· Parent-Child Hierarchies

· Unary Operators

· Many-to-Many Dimensions

· Referenced Dimensions

· Diamond Shaped Hierarchies

· Natural Hierarchies

· Junk Dimensions

· Degenerate Dimensions

· Reviewing Design Best Practices for the Source Dimension Table

· Handling Slowly Changing Dimensions


Module 11
: Query Performance Monitoring and Tuning

· Identifying Query Problems

· Checking on Partitions and Aggregations

· Implementing a Cache Warming Strategy

· Switching to Subspace Computation

· Optimizing the IIF Function

· Removing Empty Tuples

· Examining Other MDX Optimization Strategies

· Using Analysis Services Features Instead of MDX

· Finding Alternatives to Session Cubes

· Moving Calculations to the Relational Engine


Module 12
: Scale-Out Querying with Read-Only Databases

· Considering Reasons for Scale-Out Querying

· Examining the Options for Scaling Out

· Configuring the Architecture

· Balancing the Load Across Multiple Query Servers

· Updating the Analysis Services Database

· Copying the Database Files

· Optimizing the Storage System and I/O

 

 

SSAS Maestro Program Course Assessment

Note that this course is unlike many other Analysis Services courses. Attendees that satisfactorily pass the assessment will be endorsed by Microsoft, and subsequently promoted within the SQL Server community and recommended for suitable engagements by SQL CAT team.

 

Assessment will be based on:

  • 100% attendance during the three-day course
  • the consistent ability to contribute to, and at times lead, discussions
  • the satisfactory completion of all labs, and
  • successfully passing a comprehensive exam of a ~4000 word case study due 30 days after the course completion that demonstrates an applied deep understanding of the product and technology, and strong written communication skills

 

SSAS Maestro Program Benefits

Upon successful completion of this course, assessment and case study evaluation by industry experts, attendees will:

  • Be a part of the elite group of SSAS Maestros
  • Help Microsoft accounts teams deploy highly complex/highly large SSAS projects
  • Have access to webcasts and Q&A sessions on the learnings some of the most complex SSAS implementations directly from SQLCAT and the Analysis Services team
  • Be showcased on a special section within the Microsoft SQL Server Web page for the SSAS Maestros