How to create XML sitemap programmatically in c#

Question

I am building an website. Now I want to create its xml site map like google site map. But I want to create it programmatically using C#.

Can anybody tell me how I can access the root directory on the web server using base url of my website get all the pages list into a string list?

Answer 1

You can have a look at this NuGet package (that support .Net and .Net Core) https://www.nuget.org/packages/xsitemap/

class Program
{
    static void Main(string[] args)
    {
        var sitemap = new Sitemap();

        sitemap.Add(new Url
        {
            ChangeFrequency = ChangeFrequency.Daily,
            Location = "http://www.example.com",
            Priority = 0.5,
            TimeStamp = DateTime.Now
        });

        sitemap.Add(CreateUrl("http://www.example.com/link1"));
        sitemap.Add(CreateUrl("http://www.example.com/link2"));
        sitemap.Add(CreateUrl("http://www.example.com/link3"));
        sitemap.Add(CreateUrl("http://www.example.com/link4"));
        sitemap.Add(CreateUrl("http://www.example.com/link5"));


        //Save sitemap structure to file
        sitemap.Save(@"d:\www\example.com\sitemap.xml");

        //Split a large list into pieces and store in a directory
        sitemap.SaveToDirectory(@"d:\www\example.com\sitemaps");

        //Get xml-content of file
        Console.Write(sitemap.ToXml());

        Console.ReadKey();
    }

    private static Url CreateUrl(string url)
    {
        return new Url
        {
            ChangeFrequency = ChangeFrequency.Daily,
            Location = url,
            Priority = 0.5,
            TimeStamp = DateTime.Now
        };
    }
}

The original project is available here https://github.com/ernado-x/X.Web.Sitemap

Et voilà ! :)

Answer 2

I have made this library which makes it quite easy to create google sitemaps from a class or list a of urls.

https://github.com/aseemgautam/google-sitemap

Answer 3

If your your site pages are linked to each other and an orginary user can surf all of them (having necessary links in pages content), it is possible to create a list of site's webpages recursively and put it to an xml file (adhering to standards of sitemap protocol) Code snippet of url list generator from working app:

...
   new_urls.Add(BaseUrl);  //first url
   do
   {
      List hrefs=new List();
      foreach (var url in new_urls)
      {
         string text =await _loader.Get(url);
         if (string.IsNullOrEmpty(text)) continue;

         visited.Add(url);
         List meta=Parser.GetAHrefs(text).Distinct().ToList();  //getting list of links
         Parser.Normalize(Domain,url,ref meta);
         if (Exclude)  //option to exclude query from url
             meta = meta.Select(u => u.Contains('?') ? u.Split('?')[0] : u).ToList();
         hrefs.AddRange(meta);
         hrefs = hrefs.Distinct().ToList();
       }
       new_urls = hrefs.Except(visited).ToList();   //excluding visited pages
    }
    while (new_urls.Count != 0);
...

Plain text to xml parsing method:

public void Save(string path)
        {
            string doc = "<?xml version=\"1.0\" encoding=\"UTF - 8\"?>";

            doc += OpenTag("urlset", "xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\"");

            if (UseOpt)
            {
                foreach (var url in Urls)
                {
                    doc += OpenTag("url");
                    doc += Tag("loc", url);
                    doc += Tag("lastmod", LastMode);
                    doc += Tag("changefreq", Changefreq);
                    doc += Tag("priority", Priority);
                    doc += CloseTag("url");
                }
            }
            else
            {
                foreach(var url in Urls)
                {
                    doc += OpenTag("url");
                    doc += Tag("loc", url);
                    doc += CloseTag("url");
                }
            }

            doc += CloseTag("urlset");

            File.WriteAllText(path,doc);
        }