SharePoint - ULS Logs

SharePoint logging and troubleshooting using ULSViewer
In helping with troubleshooting SharePoint issues we can make use of the SharePoint Unified Logging System; or in short: ULS logs. ULS logs can provide in-depth information about the problem to a support group to enable faster resolution.

Default Location ULS log:
SPS2007 - C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\LOGS
SPS2010 - C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS
SPS2013 - C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS

ULS Verbose logging:
You can get most details by setting the ULS logging mode to Verbose or VerboseEx mode.
Note: Please be aware to only enable Verbose logging mode while reproducing the issue, because this mode will quickly fill up your hard disc.
More details can be found here:
http://technet.microsoft.com/en-us/library/ee748619.aspx (Configure diagnostic logging (SharePoint Foundation 2010))

ULS Viewer:
ULS Viewer can be used in different modes. The log can be read from log files, real time ULS log, or even clipboard. For example:
On a server that is running SharePoint 2010, run ULS Viewer. Click File, Open From, then choose ULS (This could also be done by simply press Ctrl+U). Immediately the logs will be shown in real-time. You can filter message level by click the icons in the middle. This can tell you what is going on inside SharePoint. You can filter on multiple criteria to isolate your issue.
Note: Please have a look at the Documentation link here below for more details of how to use the ULS Viewer tool. A few common ways of filtering are by using the “Correlation id” or the “Category“. Next to that, you can right-click on one of the message lines, and then select “Show Errors Only”. Or, simply do a CTRL-F (find) to search for a keyword.
Download ULS Viewer tool:
Documentation:
Thanks to the author of the article: Manlon.lam[at]Microsoft.com

SharePoint 2013 - Merge Link and Title

In a SharePoint List or Library by default, there is a column from which you can edit or view the properties of an item, e.g. column Title in the picture. This column is called Link-to-Item column.
Sharepoint
Sometimes you need to have a different column to be a Link-to-Item. Let’s say the Store IDin the Picture.
The processes is very simple but you need to have SharePoint Designer installed. SharePoint Designer is free to download from Microsoft website and is easy to install. Once you have it installed then follow the procedure below.
  • Fire up SharePoint Designer and connect to your site by clicking on Open Site and typing in the URL of your site. Will take a while to connect.
  • Now from the Navigation Pane on the left select Lists and Libraries and then select the list or library that you want to work with
  • On the summary page you should see a section called Views. In it you should see All Items. Right click on it and select Edit File in Advanced Mode
  • Search for the tag . Once you find it you should see a few tags underneath it.
SharePoint Code
  • Find the one that has the name of the column you are looking for. In our case I am looking for Store ID. You should now add LinkToItem=”TRUE” after Name attribute so the final product should look something like this;
SharePoint Code
  • Save your view and now you can either preview it or test it in your browser. You should now have something like this where Store ID is now a link.
Sharepoint
  • If you also want the menu drop down to appear on the Store ID you should also add this attribute ListItemMenu="TRUE"
SharePoint Code
Remember if it didn’t work for you straight away give it a few minutes time.
Some people have reported that changing “LinkToItem” to “linkToItem” does the trick for them. Also I’ve seen that when you add both attributes ListItemMenu="TRUE" LinkToItem=”TRUE” it starts working then you can remove the ListItemMenu="TRUE".
Hope this has helped.

Machine Learning - Visualize Graph Flow

http://www.graphviz.org/

or

import os     

os.environ["PATH"] += os.pathsep + 'C:/Program Files (x86)/Graphviz2.38/bin/'

Installing Solr 6 as a Windows web service

Install on the windows platform

https://nssm.cc/download

Install Java

http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html

Goal

Get Solr 6.1 running as a Windows Service and have it recover on OutOfMemory Exceptions.

Install Solr 6.1 as a Service

  1. Download NSSM
  2. Extract nssm.exe somewhere
  3. Create solr_start_6.1.0.cmd (this is a Windows Command file that does all of my configuration--yours will definitely be different):
    C:\apache\solr-6.1.0\bin\solr start -f -h node3 -c -p 8983 -z "zk1:2181,zk2:2181,zk3:2181" -m 128m -s C:\apache\solr-6-cores
    NOTE: The -f to run the script in the foreground.  I set the JVM heap size to 128 MB (we want this thing to crash and burn to test out OutOfMemoryException restart!).
  4. Test your script to make sure it starts solr and you can access your Solr Admin UI in a web browser
  5. Open a command window and navigate to your nssm.exe directory
  6. nssm.exe install "Apache - Solr 6.1"
    1. Application Tab
      1. Path: Select your solr_start_6.1.0.cmd from earlier
      2. Startup Directory: set it to the directory containing your script (should populate by default)
    2. Details Tab
      1. Display Name: Apache - Solr 6.1
      2. Startup type: Automatic
    3. Log on Tab
      1. Make sure you specify an account that has administrator-level permissions (Use your account if you're stuck here--but make sure to set it to something production-worthy later)
    4. I/O Tab
      1. I/O Redirection
        1. Output (stdout): Set this to something like path\to\my\solr\cmd\script\dir\solr-6.1.0-out.txt
        2. Error (stderr): path\to\my\solr\cmd\script\dir\solr-6.1.0-error.txt
      2. File rotation
        1. Check Rotate files
        2. Check Rotate while service is running
        3. Restrict rotation to files bigger than: (use common sense here, I did 5 MB, so 5242880 went into the box)
    5. Click Install Service
  7. Open Component Services and select Apache - Solr 6.1
  8. Start the service
  9. Validate that it came up by going to your Admin UI webpage

    Make Solr Service respond to Out Of Memory Exceptions

    1. Navigate to this JIRA ticket
    2. Download oom_win.cmd and place it in your solr\bin directory next to solr.cmd
    3. Open solr.cmd in a text editor
    4. Find all the places where the script starts the server:
      1. Search for /solr_gc.log
    5. Immediately after /solr_gc.log, paste the following:
      -XX:OnOutOfMemoryError="%SOLR_SERVER_DIR%\..\bin\oom_win.cmd %SOLR_PORT% !SOLR_LOGS_DIR!"
      1. I had to replace two lines.  NOTE that this is just the manual way of applying the patch file associated with the JIRA ticket above.  If you want, apply it however you want.
    6. Now that we've made our changes, go ahead and restart our new Solr 6.1 service so it knows to kill the process on OutOfMemory errors.
    7. To force an OutOfMemoryError, query *:* and return 1000000 rows
      1. If you have a decent amount of content, this should force an OutOfMemory exception.  If you don't have a lot of content, do whatever you can to make it do a lot of memory-intensive work.  Perhaps consider lowering the JVM memory, too.
      2. You should see the web server go offline temporarily and then come back online
    8. Now that you've seen it restart and come back online, let's give the JVM a good amount of RAM so that it doesn't run out of memory every other request.  Just edit your solr_start_6.1.0.cmd file and change the -m 128m to -m 4g (128 MB to 4 GB)
    9. Save and restart the service
    10. Confirm that you have the new amount of RAM for the JVM by visiting the Dashboard tab in the Admin UI

    Logs

    When the OutOfMemory Killer runs, it generates a log file in the normal log directory.  Navigate to that directory and you should see a file that looks something like: solr_oom_killer-9000-2016-07-06_13_59_39.  Now you can know when this script runs and hopefully anticipate it in the future or make changes to not get it.


    Weak Diffie-Hellman and the Logjam Attack - Revoke from Web Server

    Diffie-Hellman key exchange is a popular cryptographic algorithm that allows Internet protocols to agree on a shared key and negotiate a secure connection. It is fundamental to many protocols including HTTPS, SSH, IPsec, SMTPS, and protocols that rely on TLS.
    We have uncovered several weaknesses in how Diffie-Hellman key exchange has been deployed:
    1. Logjam attack against the TLS protocol. The Logjam attack allows a man-in-the-middle attacker to downgrade vulnerable TLS connections to 512-bit export-grade cryptography. This allows the attacker to read and modify any data passed over the connection. The attack is reminiscent of the FREAK attack, but is due to a flaw in the TLS protocol rather than an implementation vulnerability, and attacks a Diffie-Hellman key exchange rather than an RSA key exchange. The attack affects any server that supports DHE_EXPORTciphers, and affects all modern web browsers. 8.4% of the Top 1 Million domains were initially vulnerable.
    2. Threats from state-level adversaries. Millions of HTTPS, SSH, and VPN servers all use the same prime numbers for Diffie-Hellman key exchange. Practitioners believed this was safe as long as new key exchange messages were generated for every connection. However, the first step in the number field sieve—the most efficient algorithm for breaking a Diffie-Hellman connection—is dependent only on this prime. After this first step, an attacker can quickly break individual connections.
      We carried out this computation against the most common 512-bit prime used for TLS and demonstrate that the Logjam attack can be used to downgrade connections to 80% of TLS servers supporting DHE_EXPORT. We further estimate that an academic team can break a 768-bit prime and that a nation-state can break a 1024-bit prime. Breaking the single, most common 1024-bit prime used by web servers would allow passive eavesdropping on connections to 18% of the Top 1 Million HTTPS domains. A second prime would allow passive decryption of connections to 66% of VPN servers and 26% of SSH servers. A close reading of published NSA leaks shows that the agency's attacks on VPNs are consistent with having achieved such a break.

    Full Technical Paper

    Imperfect Forward Secrecy: How Diffie-Hellman Fails in Practice
    David Adrian, Karthikeyan Bhargavan, Zakir Durumeric, Pierrick Gaudry, Matthew Green, J. Alex Halderman, Nadia Heninger, Drew Springall, Emmanuel Thomé, Luke Valenta, Benjamin VanderSloot, Eric Wustrow, Santiago Zanella-Béguelin, and Paul Zimmermann
    22nd ACM Conference on Computer and Communications Security (CCS ’15), Denver, CO, October 2015
    Best Paper Award Winner
    More: Full paper | Talk slides | Bibtex

    Additional Information

    Contact the Team

    This study was performed by computer scientists at CNRS, Inria Nancy-Grand Est, Inria Paris-Rocquencourt, Microsoft Research, Johns Hopkins University, University of Michigan, and the University of Pennsylvania: David AdrianKarthikeyan BhargavanZakir Durumeric,Pierrick GaudryMatthew GreenJ. Alex HaldermanNadia HeningerDrew SpringallEmmanuel ThoméLuke ValentaBenjamin VanderSloot,Eric WustrowSantiago Zanella-Beguelin, and Paul Zimmermann. The team can be contacted at weakdh-team@umich.edu.

    Who is Affected?

    Websites, mail servers, and other TLS-dependent services that support DHE_EXPORT ciphers are at risk for the Logjam attack. We use Internet-wide scanning to measure who is vulnerable.
    ProtocolVulnerable to Logjam
    HTTPS — Top 1 Million Domains8.4%
    HTTPS — Browser Trusted Sites3.4%
    SMTP+StartTLS — IPv4 Address Space14.8%
    POP3S — IPv4 Address Space8.9%
    IMAPS — IPv4 Address Space8.4%
    Websites that use one of a few commonly shared 1024-bit Diffie-Hellman groups may be susceptible to passive eavesdropping from an attacker with nation-state resources. Here, we show how various protocols would be affected if a single 1024-bit group were broken in each protocol, assuming a typical up-to-date client (e.g., most recent version of OpenSSH or up-to-date installation of Chrome).
    Vulnerable if most common 1024-bit group is broken
    HTTPS — Top 1 Million Domains17.9%
    HTTPS — Browser Trusted Sites6.6%
    SSH — IPv4 Address Space25.7%
    IKEv1 (IPsec VPNs) — IPv4 Address Space66.1%

    What Should I Do?

    If you run a server…

    If you have a web or mail server, you should disable support for export cipher suites and use a 2048-bit Diffie-Hellman group. We have published a Guide to Deploying Diffie-Hellman for TLS with step-by-step instructions. If you use SSH, you should upgrade both your server and client installations to the most recent version of OpenSSH, which prefers Elliptic-Curve Diffie-Hellman Key Exchange.

    If you use a browser…

    Make sure you have the most recent version of your browser installed, and check for updates frequently. Google Chrome (including Android Browser), Mozilla Firefox, Microsoft Internet Explorer, and Apple Safari are all deploying fixes for the Logjam attack.

    If you’re a sysadmin or developer …


    Make sure any TLS libraries you use are up-to-date, that servers you maintain use 2048-bit or larger primes, and that clients you maintain reject Diffie-Hellman primes smaller than 1024-bit.

    Digital, Technology, and Innovation - DTI

    Digital, Technology, and Innovation (DTI) is a broad term that encompasses the use of digital technologies to improve business processes, products, and services. It is a rapidly growing field that is transforming the way businesses operate.

    Here are some of the key trends in DTI:

    • Artificial intelligence (AI): AI is a rapidly evolving field that is being used to automate tasks, improve decision-making, and create new products and services.
    • Big data: Big data is the collection of large and complex data sets that can be analyzed to gain insights into customer behavior, market trends, and other areas.
    • Blockchain: Blockchain is a secure and transparent way to record transactions. It is being used to create new financial products and services, as well as to improve supply chain management.
    • The Internet of Things (IoT): The IoT is the network of physical devices that are connected to the internet. These devices can collect and share data, which can be used to improve efficiency, safety, and comfort.
    • Cybersecurity: Cybersecurity is the protection of computer systems and networks from unauthorized access, use, disclosure, disruption, modification, or destruction. It is essential to protect sensitive data and prevent cyberattacks.

    DTI is a rapidly growing field that is transforming the way businesses operate. By understanding the key trends in DTI, businesses can stay ahead of the competition and improve their operations.

    Here are some of the benefits of DTI:

    • Improved efficiency: DTI can help businesses to improve efficiency by automating tasks, streamlining processes, and making better decisions.
    • Increased customer satisfaction: DTI can help businesses to increase customer satisfaction by providing personalized experiences, improving customer service, and making it easier for customers to do business with the company.
    • New product and service development: DTI can help businesses to develop new products and services by providing insights into customer needs and trends.
    • Risk mitigation: DTI can help businesses to mitigate risk by identifying and addressing potential problems.
    • Cost savings: DTI can help businesses to save money by automating tasks, reducing errors, and improving efficiency.

    DTI is a valuable tool that can help businesses to improve their operations and stay ahead of the competition. If you are interested in learning more about DTI, I recommend that you take some time to research the latest trends and developments.

    SQL Updating ID with Row Number

    with toupdate as (
          select t.*, row_number() over (partition by Employee_ID order by DOH) as seqnum
          from Employee_Immunizations t
         )
    update toupdate
         set ID = seqnum;