Convert (archive) SharePoint Pages to PDF

Problem-Scenario:

During our SharePoint 2010  to SharePoint Online migration, i was asked to archive around 1000+ article pages (used as company newsletters) as PDF and store in SharePoint Online site. If i could archive those pages as PDF, there was no need to keep all those sites. So it was also a good opportunity to cleanup our environment before migration

Solution:

First thing that came to my mind was to print from Internet Explorer but the results were horrendous. It would take 7-8 pages to print one article page.

I searched for third party tools and found PDFCreator and WkhtmltoPdf tools.

Option 1) automating using wkhtmltopdf tool

Observation:

1. links are clickable
2. banner is coming
3. photos are not coming in most cases
4. colors were better than option 2)
5. Takes 6-7 pages to print one sharepoint article page

Option 2) automating using PDFCreator + Powershell

Observation:

a.       links are showing up but aren't clickable

b.       banner is not coming

c.       photos are coming
d.   Takes 6-7 pages to print one sharepoint article page

Still both the above options were taking few pages to print just one article page. This wasn't ideal since users won't be able to make any sense out of reading these PDFs. I had to come up with something better. Thanks to Stackexchange site, i read about a post asking to try command line of chrome.

To test, I tried printing using chrome and it works perfectly. It takes only one page to print one article page, has better colors, all links are clickable, all images gets printed.

Using the headless and disabling GPU options did the trick.

This obviously needs Chrome browser to be installed on the server. Since i wasn't allowed to install Chrome on the SharePoint server, i copied the Google folder from my desktop-program files and pasted it on SP server. That was all it took to make the command work. In the powershell, you have to mention the path to chrome.exe as you will see in the code below:

# This script is intended to work on SharePoint 2010. For higher versions,
# just use CSOM
# Load Microsoft.SharePoint
# Open your SharePoint web

[void][System.reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") 

$site = new-object Microsoft.SharePoint.SPSite("http://your absolute sharepoint 2010 web Url where newsletters/article pages reside")                                                        

$web = $site.openweb()

write-host $web.Title

# Get all publishing pages

# Since there were folders in library, i added Scope=Recursive

$pubweb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($web)

$query = new-object Microsoft.SharePoint.SPQuery 

$query.ViewAttributes = "Scope='Recursive'"

$pages = $pubweb.GetPublishingPages($query)    

write-host 'no of pages ' $pages.count   

    

# Loop through pages

# $newslettername: i kept it same as file name
#    Remove .aspx and change extension to .pdf
#    this also contains the desktop path where pdf will be saved

foreach ($listItem in $pages)

{

    $newslettername = ''

    $pageurl = ''

    

    $pageurl = $listItem.uri.AbsoluteUri

    

    if($pageurl -Match 'templates') {

       $newslettername = 'c:\temp\newsletters\templates\' + $listItem.name + '.pdf'    

    }

    else {   

    $newslettername = 'c:\temp\newsletters\' + $listItem.name + '.pdf'    

    }

    

    $newslettername = $newslettername -replace '.aspx',''  

    

# Give full path to chrome.exe

# Run the command for converting .aspx page i.e. newsletter i.e. article page to PDF

F:\shishir\Google\Chrome\Application\chrome.exe --headless --disable-gpu --print-to-pdf=$newslettername $pageurl

    write-host 'printed the page at' $pageurl

}

$web.Dispose()

$site.Dispose()  

Azure Devops Automate building of SharePoint Apps

Problem-Scenario:

In Azure Devops there are .net related templates available for use in build definition but there isn't anything available for building SharePoint apps

Solution:
There isn't an inbuilt template available for building SharePoint apps. So i had improvise.

After trying different options, I was able to use yml code to build pipeline for SharePoint hosted apps. 

There aren’t any inbuilt templates available for SP hosted apps.

Background: 

Our sharepoint hosted app's source code is stored in azure Repos. Repo is used as an input to build pipeline. Use minimal starter yml and then update based on the information in this blog.

Below are the main steps to remember:

Also attached is the complete yaml file at the end

Trigger

 

You can mention committing to which branch will trigger the build pipeline. You can also configure it to exclude some branches or sub-branches.

Build SP Hosted app

Ignore the script step. This just displays the internal location of directories.

Used MSBuild to build our SharePoint app Repo and output .APP file

Copy files to artifact staging directory

Copies the output for publishing
FlattenFolders: true --> Flattens the folder structure
Contents: 'app.publish/**/*.app' --> filters *.app file

Publishing the artifact

Where does the artifacts go? Or how to access it?

Open build result & look at snapshot below:

The folder structure can be changed if needed.

This is saved for each build that is run and is retained as per our retention policy.

This artifact, in turn, will directly consumed by RELEASE pipeline which can either be triggered automatically or manually.

Below is the entire .yml file for your ease of use.

# Starter pipeline

# Start with a minimal pipeline that you can customize to build and deploy your code.

# Add steps that build, run tests, deploy, and more:

# https://aka.ms/yaml

trigger:

- master

pool:

vmImage: 'windows-latest'

variables:

solution: '**/*.sln'

buildPlatform: 'Any CPU'

buildConfiguration: 'Release'

steps:

- script: echo Hello, world!DA1S $(Build.SourcesDirectory) and agent da1 is $(Agent.BuildDirectory) AND $(System.DefaultWorkingDirectory)/bin/

displayName: 'Run a one-line script'

- task: MSBuild@1

inputs:

solution: '$(solution)'

msbuildArguments: '/t:Package /p:OutputPath=$(Build.SourcesDirectory)/bin/'

platform: '$(buildPlatform)'

configuration: '$(buildConfiguration)'

- task: CopyFiles@2

inputs:

SourceFolder: '$(Build.SourcesDirectory)/bin/'

Contents: '**'

TargetFolder: '$(Build.ArtifactStagingDirectory)'

- task: PublishBuildArtifacts@1

inputs:

ArtifactName: 'drop'

PathtoPublish: '$(Build.SourcesDirectory)/bin/'

How to send Outlook Meeting invite from SharePoint calendar

I like to blog on difficult issues that i face while developing.

I have built a calendar management system in SharePoint on top of SharePoint calendar.
One of the functionalities it offers to users is to add the event to outlook as a meeting. I had built 'add to outlook' i.e. outlook meeting invite functionality using downloading event in ICS file and it works fine but users have to click many times since ICS file would download, then open in outlook, save in outlook, etc. This time, I wanted to offer more efficient solution so users don't have to click multiple times.

So i came up with sending event invitation to users as an outlook meeting invite. No Clicks!
Below is how i did it:

Problem-Scenario:

Programmatically send Outlook meeting invite from SharePoint

Solution:
Here, I will cover only the part where you send Outlook meeting invite programmatically.

//Read comments for details:
//Replace Constants.variable name with your variable value.
//Replace event item column values with your item values.

public void SendMeetingInviteEmail(SPListItem currentItem)
        {
            string currentUserEmailAddress = SPContext.Current.Web.CurrentUser.Email;
            string currentUserDisplayName = SPContext.Current.Web.CurrentUser.Name;

            //get all event columns and store them in variables. Easy step, so removed code for it.
            //......//

            //Create Mail Message object with required details

            MailMessage msg = new MailMessage();
           
            msg.From = new MailAddress(Constants.EventsAdminEmailAddress,                       Constants.EventsAdminDisplayName);
            msg.To.Add(new MailAddress("steve@apple.com", "steve jobs"));
           
           // CC is optional 
            msg.CC.Add(new MailAddress("amigo@spanish.com", "Amigo Friend"));

            msg.Subject = eventTitle; //from event item
            msg.Body = eventPresenter + eventDescriptionAsText; //from event item
            msg.Headers.Add("Content-class", "urn:content-classes:calendarmessage");

/* Most important part is to create alternate views:
    You need to create two alternate views. One for calendar (mandatory) and another one for the mail      body. (optional)

     I highly recommend creating both views even if second one is optional. 
     If you don't create a       separate alternate view for mail body, it will be simple text with no    HTML whatsoever. So it will be a straight line. Users are not going to like that.
     Mail body alternate view will allow the body to be sent as html.
*/
            //Calendar Alternate View (mandatory)
            StringBuilder str = new StringBuilder();
            str.AppendLine("BEGIN:VCALENDAR");
            str.AppendLine("PRODID:-//Schedule a Meeting");//no need to change to SharePoint ID
            str.AppendLine("VERSION:2.0");
            str.AppendLine("METHOD:REQUEST");
            str.AppendLine("BEGIN:VEVENT");
            str.AppendLine("CLASS:PUBLIC");

/*All date time objects should be in UTC format else the meeting invite will break. 
   Most probably this is an Outlook meeting requirement

   eventStartTime and eventEndTime objects are converted into UTC format.
   You may ask how to do it, so here is an example:

   eventStartTime =               DateTimeOffset.Parse(currentItem[Constants.EventsListFieldStartTime].ToString()).UtcDateTime;
          
*/
            str.AppendLine(string.Format("CREATED:{0:yyyyMMddTHHmmss}", DateTime.Now));
            str.AppendLine(string.Format("DTSTART:{0:yyyyMMddTHHmmssZ}", eventStartTime));
            str.AppendLine(string.Format("DTSTAMP:{0:yyyyMMddTHHmmssZ}", DateTime.UtcNow));
            str.AppendLine(string.Format("DTEND:{0:yyyyMMddTHHmmssZ}", eventEndTime));
            str.AppendLine(string.Format("LOCATION:{0}", eventLocation));
            str.AppendLine(string.Format("UID:{0}", Guid.NewGuid().ToString()));
            str.AppendLine(string.Format("DESCRIPTION:{0}", msg.Body));
            str.AppendLine(string.Format("X-ALT-DESC;FMTTYPE=text/html:{0}", msg.Body));
            str.AppendLine(string.Format("SUMMARY;LANGUAGE=en-us: Scheduled for {0}", msg.Subject));
            str.AppendLine(string.Format("ORGANIZER:MAILTO:{0}", msg.From.Address));

         
/*
Below is how we create meeting alert for user.
              TRIGGER:-PT15M
It means that user alert will activate 15 minutes before the meeting starting time. 
You can change it based on your requirements   
*/
            str.AppendLine("BEGIN:VALARM");
            str.AppendLine("TRIGGER:-PT15M");
            str.AppendLine("ACTION:DISPLAY");
            str.AppendLine("DESCRIPTION:Reminder");
            str.AppendLine("END:VALARM");
            str.AppendLine("END:VEVENT");
            str.AppendLine("END:VCALENDAR");

            string eventsHomeUrl = SPContext.Current.Site.Url + Constants.EventsWebUrl;

            SmtpClient smtpclient = new SmtpClient(GetSMTPHostName(eventsHomeUrl));
            smtpclient.Credentials = CredentialCache.DefaultNetworkCredentials;

//add mail body HTML type alternate view
            ContentType HtmlCtype = new ContentType("text/html");

//eventPresenter and eventDescription are event item column values
            string bodyHtml = GetBodyHtml(eventPresenter + "<br>" + eventDescription);

            AlternateView HtmlView = AlternateView.CreateAlternateViewFromString(bodyHtml, HtmlCtype);
            msg.AlternateViews.Add(HtmlView);
           

//REMEMBER to add Calendar view last / all other views before it else Outlook invite won't work
            ContentType contype = new ContentType("text/calendar");
            contype.Parameters.Add("method", "REQUEST");

            AlternateView avCal = AlternateView.CreateAlternateViewFromString(str.ToString(), contype);
            msg.AlternateViews.Add(avCal);
            smtpclient.Send(msg);
        }

/* 
Get SMTP host name from SharePoint outgoing e-mail settings. Good practice to set it at web application level.
*/

private static string GetSMTPHostName(string siteUrl)
        {
            using (SPSite site = new SPSite(siteUrl))
            {
                //Get the SMTP host name from “Outgoing e-mail settings”
                return site.WebApplication.OutboundMailServiceInstance.Parent.Name;
            }
        }

/*
Last but not least is how to create mail body. It is self explanatory. See below:
*/

private static string GetBodyHtml(string body)
        {
             string bodyHtml =
                 @"<!DOCTYPE HTML PUBLIC ""-//W3C//DTD HTML 3.2//EN"">"
                 + @"<HTML>"
                 + @"<HEAD>"
                 + @"<META HTTP-EQUIV=""Content-Type"" CONTENT=""text/html; charset=utf-8"">"
                 + @"<META NAME=""Generator"" CONTENT=""MS Exchange Server version 6.5.7652.24"">"
                 //+ @"<TITLE>{0}</TITLE>"
                 + @"</HEAD>"
                 + @"<BODY>"
                 + @"<BR>"
                 + @"{0}"
                 + @"</BODY>"
                 + @"</HTML>";

             return String.Format(bodyHtml, body);
        }

Do comment if this blog has helped you!

C#.net - Find and Replace in large files

Problem-Scenario: When we are working on large files (e.g. > 1GB) and you have to do simple operation like Replace a string with another string.
regex.replace or string.replace or using xmldom object or LINQ doesn't work! And there are no free tools in market which does it without throwing up in middle.
  
Solution:
Well the solution is as simple as it can get. Just use StreamReader & StreamWriter. These don't load file in memory but streams through your text or xml file byte by byte

Example: (Change the required parameter values for changing a site's site definition)

            /// Replaces text in a file.
            ///

Path of the text file.
            ///Text to search for.
            ///Text to replace the search text.
            
public void ReplaceInFile(string SourcefilePath, string DestfilePath)
            {
               string data;
                if( !(SourcefilePath.Contains(".xml")) )
                {
                    Console.WriteLine("Please specify Manifest.xml path.. Filename is missing");
                    return;
                }
               if(File.Exists(SourcefilePath) == false)
               {
                   Console.WriteLine("File doesn't exist at the specified path\n");
                   return;
               }
              

            StreamReader streamReader = new StreamReader(SourcefilePath);
            StreamWriter streamWriter = new StreamWriter(DestfilePath);

            while (streamReader.Peek() >= 0)
            {

                data = streamReader.ReadLine();

                //**********************************
                // Strings for changing the Configuration IDs
                string OldConfig1 = @"Configuration=""0""";
                string NewConfig = @"Configuration=""2""";

                //-1. Webtemplate="InsideCustompublishingWorkflow - Config as -1 change it to 2
                string searchtext23 = @"WebTemplate=""InsideCustompublishingWorkflow""";

                if (data.Contains(searchtext23) == true)
                {
                    //change the configuration
                    data = data.Replace(OldConfig1, NewConfig);
                 }

           
                //CHANGE THE SITE TEMPLATE NAME
                // 1. WebTemplate="INSIDECustomPUBLISHING" - Aold
                string searchtext1 = @"WebTemplate=""INSIDECustomPUBLISHING""";
                string replacetext1 = @"WebTemplate=""INSIDECustomPUBLISHINGnew""";

                if (data.Contains(searchtext1) == true)
                {
                    //change the configuration
                    data = data.Replace(OldConfig1,NewConfig);
                    data = data.Replace(searchtext1, replacetext1);
                   
                }
               

                //CHANGE THE SITE TEMPLATE SETUPPATH
                //INSIDECustomPUBLISHING - Aold //SetupPath="SiteTemplates\INSIDECustomPUBLISHING"
                string searchtext4 = @"SetupPath=""SiteTemplates\INSIDECustomPUBLISHING\";
                string replacetext4 = @"SetupPath=""SiteTemplates\INSIDECustomPUBLISHINGnew\";
                data = data.Replace(searchtext4, replacetext4);


                //Write the data on .xmlnew file
                streamWriter.WriteLine (data);

            }

            streamReader.Close();
            streamWriter.Close();
}

 

How to change an existing site's site definition?

Problem-Scenario:

I faced this problem in one of my projects. There were many custom site definitions beings used in the organization & few were all messed up; so they wanted me to change the site definition of all the existing sites to just one custom site definition.

Solution:
Now there are two supported ways of changing a site's site definition. Yeah! you heard it right. It is supported by Microsoft.

First way is using the sharepoint deployment API. You can refer to Stefan Gobner's blog on this:
http://blogs.technet.com/stefan_gossner/archive/2007/08/30/deep-dive-into-the-sharepoint-content-deployment-and-migration-api-part-4.aspx
I personally found it difficult, so i analyzed the manifest.xml & did lot of testing to find another simpler solution which doesn't use SharePoint API but does the same thing.

Note: you cannot save site as template (.Stp) because it has limitation of maximum 10MB size which makes the approach useless

The second way is to manipulate the manifest.xml directly
Steps:

1) Find the template name, setuppath, site definition ID, Configuration ID of all the site definitions which needs to be changed as well as of the new site definition. 

For that go to

C:\\Program Files\\Common Files\\Microsoft Shared\\web server extensions\\12\\TEMPLATE\\1033\\XML  open the webtemp.xml of your site definition. after couple of lines, you will see:

"CustomsiteDefitionName" is template name and "50" is template ID or sitedefinition ID

What is the configuration ID for the configuration we want to use?

Our sites are using "Custom team Site", so our configuration id is "0"

In the file, you will also see SetupPath=""SiteTemplates\PUBLISHING\"; for publishing site for example.

Note down these for all site definitions in question.

2) Export the site whose site definition you want to change

stsadm -o export  -url   -filename  [-includeusersecurity]  [-versions] <1-4> -nofilecompression   [-quiet]

This stsadm command will export the site, including it’s subsites, in non compressed fashion. The result will be lot of .dat or data files, few xml – important ones -> Manifest.Xml & Requirements.xml

3) Open Manifest.xml &Requirements.xml & change TemplateName, TemplateID,ConfigurationID & Setuppath of the current site definition to the new sitedefinition's one

That's it & you are almost done.

4) Import this site back to its place. You can delete or overwrite the previous version..upto you. I would suggest taking backup first.

5) Njoy! your site's site definition has been changed.

Note: If your manifest.xml grows bigger (like more than 1GB), any DOM manipulation won't work on it. All DOM ways of opening big files fail since it loads everything in memory. Read my blog on how to handle it...

Get GUID of SPListItem

Category: Working with SPListItem

Scenario: I was trying to check a column value in splistitem in an Asynchronous event handler.
Since it is asynchronous, the values had already changed & the before state is unaccessible through properties object.

Problem: How to get GUID of a SPListItem so that you can access list item values before the event handler updates it

Solution: use SPListItem.UniqueId

Example:

Guid SiteGuid = (Guid)properties.SiteId;
string weburl = (string)properties.WebUrl;
using (SPSite tempsite = new SPSite(SiteGuid))
{
using (SPWeb tempweb = tempsite.OpenWeb(()))
{
SPList templist = tempweb.Lists[];
Guid templistitemID = (Guid)properties.ListItem.UniqueId;  //get GUID of splistitem
SPListItem templistitem = templist.Items[templistitemID]; //for accessing splistitem values
}
}

Two Level Grouping in XSLT 1.0 in SharePoint Content Query Webpart

Category: XSLT1.0 & SharePoint's Content Query WebPart

Problem/ Scenario: 

We have to display data in a Content Query WebPart from a SharePoint list & apply two level grouping to it. Below is an example of two level grouping.

We have three columns in the list namely FoundationName, CauseName & FundName. We need to group the data first by FoundationName & then by Causename. Below is an example of how the data should look like in the end.

Foundation1

            Cause1

                        Fund1

                        Fund2

                        Fund3

            Cause2

                        Fund4

Foundation2

            Cause1

                        Fund5

            Cause3

                        Fund6

Solution:

Step 1:

In order to understand how SharePoint renders the Content Query Web Part, let's look at the Raw Xml

Refer to: http://www.sharepointblogs.com/radi/archive/2009/03/17/content-query-web-part-getting-a-full-dump-of-the-raw-xml.aspx

Below is the raw Xml I got when i connected to my list:

<dsQueryResponse>

    <Rows>

      <Row ListId="40EB7A9E-6F40-479B-B510-D321E60A47E7" WebId="72FAC64E-A65A-4F66-81CA-0240777D7FB2" ID="2" _x007B_fa564e0f_="Brain" FileRef="Site1/Lists/CustomList/2_.000" _x002D_9fedbadb6ce1_x007D_="2" Modified="2009-06-22 16:58:20" Author="System Account" Editor="System Account" Created="2009-06-17 18:01:40" PublishingRollupImage="" _Level="1" Comments="" Title="Brain" Description="<div>this is leader text for Brain</div>" NewFeatured="Featured" SiteUrl="http://Server/site1" ContextScriptID="ctxg_588a882e_6730_4693_b3f0_2bbf668aa8021" ItemIndex="2" ItemContentType="" ItemContentTypeID="" ItemApplicationName="" ItemOpenMethod="" ItemCheckedOut="0" ItemCheckedOutUserID="" ItemIconUrl="blank.GIF" ItemExtension="" DocumentFolder="Site1/Lists/CustomList" ItemUrl="/Site1/Lists/CustomList/2_.000" DocType="" CopySource="" ModerationStatus="0" DocEditCommand="" LinkUrl="http://Server/site1/Lists/CustomList /2_.000" PubDate="Mon, 22 Jun 2009 16:58:20 GMT" ImageUrl="" ImageUrlAltText="" Style="CustomStyle1 " GroupStyle="DefaultHeader" __begincolumn="True" __begingroup="False" />

  </dsQueryResponse>

This will give you an idea on how SharePoint renders XML of Content Query Web part

Step 2:

I am assuming here that you know about ContentQuerymain.xsl, Header.xsl & ItemStyles.xsl. Refer to http://msdn.microsoft.com/en-us/library/bb447557.aspx

Since we have to group our data, we will be coding in ContentQueryMain.xsl. (Note: It is a good practice to create your own custom ContentQueryMain file & make your CQWP reference to this file.)

In ContentQueryMain.xsl, replace

<xsl:template match="/">

    <xsl:call-template name="OuterTemplate" />

  </xsl:template>

with

<xsl:template match="/">

    <xsl:call-template name="OurCustomTemplate" />

  </xsl:template>

Step 3:

We will be using Keys for creating two level grouping. Two keys in our case:

  <xsl:key name="foundationsKey" match="Row" use="@AssociatedFoundation" />

  <xsl:key name="associatedCause" match="Row" use="@AssociatedCause" />

Put these keys just before the code mentioned in Step 2:  in ContentQueryMain.xsl

Here @AssociatedFoundation & @AssociatedCause are internal column names which will be used for grouping

Step 4:

Creating the custom template:
  <xsl:template name=" OurCustomTemplate ">

<xsl:for-each select="/dsQueryResponse/Rows/Row[generate-id(.)=generate-id(key('foundationsKey', @AssociatedFoundation))]">

/*Going upto the <Row> node in the xml & getting Foundation Name*/

<div>

<xsl:value-of select="@AssociatedFoundation" />

</div>

/*Display the Foundation name*/

        <xsl:variable name="thisFoundation" select="@AssociatedFoundation"/>

/*Storing Foundation name in a variable 'thisFoundation'  so that we can use it in our inner loop*/

        <xsl:for-each select="../Row[generate-id() = generate-id(key('associatedCause', @AssociatedCause)[@AssociatedFoundation = $thisFoundation][1])]">

/*For @AssociatedFoundation = the current foundation name, get the associated causes & store in @AssociatedCause */

          <xsl:sort select="@AssociatedCause" /> /*Sort based on AssociatedCause*/

                                                <xsl:for-each select="key('associatedCause', @AssociatedCause)[@AssociatedFoundation = $thisFoundation]">

                                            <xsl:if test="position() = 1">

                                                                                    <xsl:value-of select="@AssociatedCause" />

/*Display the cause when position()=1 else it will display CauseName for every Fund Name*/                       

                    </xsl:if>

                                                        <xsl:value-of select="@FundName" />

/*For the particular foundation name & cause name, display the corresponding Fund Name*/

                                                                                                                    

                </xsl:for-each>

                                    </xsl:for-each>

            </xsl:for-each>

  </xsl:template>

Final Notes: Similar way multi level grouping can be done