Table of Contents

Purpose

Troubleshooting URL filtering and cloud lookup issues

1.Check if the URL filtering is configured correctly for category/whitelist/blacklist/reputation.

2.Check if the website falls under the category configured

3.Check if in the expected security policy URL filter is attached

4.Commands to collect the statistics and log files

5.Commands to collect the statistics if cloud lookup is enabled.

6. Contact support.

7. Frequently Asked Questions.

Purpose:

In this document, we will see how to troubleshoot issues related to URL filtering and cloud lookup

1. Check if the URL filtering is configured correctly for Category / Blacklist / Whitelist / Reputation

For the URL filtering profile, browsed URL would be evaluated as per below sequence:

a) Blacklist

b) Whitelist

c) Category and Reputation. Here whatever action gets matched, out of that the action with most severity would be preferred.  

Also in a tie situation between custom category and user-defined category, the action with most severity would be preferred.

Let us consider an example where category based action is configured.

Here we have created a URL filter to allow only shopping websites else everything is kept as blocked:

2. Check if the website falls under the category configured.

Suppose we want users to access amazon.com but not google.com, then amazon.com should fall under “shopping” category and google.com should fall under category other than shopping, below is how you can verify:

Hence amazon.com falls under shopping category and google.com falls under search_engines category.

Hence according to our configuration we should be able to browse amazon.com and not google.com.

3. Check if the URL filter is correctly attached to security policy.

Below is the security policy configuration to achieve the URL filtering applied:

As per the above, the traffic hitting flexVNF from LAN-zone would get evaluated by allow_shopping access policy where we have attached the URL filter. 

4. Commands to collect the statistics and confirm if expected policies are hit:

Clear the URLf and access policy stats before performing the test:

request clear statistics security urlf-profile org org-name ORG-1 profile-name <urlf-profile-name>

request clear statistics security access-policy org org-name ORG-1 rule-name <rule-name>

Now browse shopping website for eg: amazon.com and look for parameters such as hit count, total_url_category_actions, total_block_actions:

In the above screenshot we see that correct access policy is hit and also url-filter is hit.

While troubleshooting such issues the hit count for the respective urlf-profile hits and counters/access-policies/cloud-lookup hits and counters should be taken into consideration.

Suppose here we are looking for category based urlf filter, hence the counters we need to take into consideration are “total_hits”, “total_url_category_actions”, “total_url_p_category_actions” (for pre-defined categories), “total_allow_actions”

Accordingly if there has been blacklist, whitelist, or reputation based actions are configured the counters needs to be looked into accordingly if they are hit correctly for the traffic hitting FlexVNF. Also the actions configured would be having the hit counts accordingly.

As shown in the above screenshot, we had set the action as “allow” for “shopping” category and hence the counter “total_allow_actions” got increased.

 The above screenshot is the output of the session for the amazon.com accessed.

 Here the access policy should be matched where the expected URL filter is attached.

 As per below screenshot we were able to browse amazon.com:

 Now we will browse google.com and hence the block counters should increase after clearing the stats:

Moreover, analytics can also be used to check the URLf actions for blocked URL, below screenshot would be helpful from the analytics part:

Also urlf module can be kept in debug and the debug logs can be seen in service.log file. It is present at location /var/log/versa/versa-service.log. Debug should only be enabled for troubleshooting purposes and deleted once troubleshooting is done

Below command can be used to enable and delete debug from configure terminal:

set debug urlf all-flags level all

delete debug urlf

 5. Commands to collect the statistics if cloud lookup is enabled.

If cloud lookup is enabled then first clear the below statistics and then collect them by accessing the website to confirm if cloud profile is hit and counters increase:

request clear statistics security urlf-profile org org-name <org-name> profile name <urlf-profile-name>

request clear statistics security urlf-cloud-lookup org org-name <org-name>

show orgs org-services <org-name> security url-filtering statistics cloud-lookup

 Once the URL is browsed the cloud-lookup counters should increase, below output is for the cloud lookup     mode being synchronous, if the lookup mode was asynchronous then async counters would have increased:

6. Contact Support

Please reach out to support with all the above outputs and screenshots.

7. Frequently Asked Questions

This section provides answers to frequently asked questions (FAQs).

i) Does the Versa-Director GUI use both cloudlookup and Predefined spack lookup for showing results here?

[Answer] The Versa-Director "Look Up URL" feature only uses Predefined spack, but there can be an instance where, if there was a cloud lookup that got triggered due to transit traffic, and based on the caching, the UI can show cached results.

Below is an example: 

(1) This is a regular lookup that happens in the Director-GUI.

VOS-cli:

admin@Branch-cli> request orgs org-services Tenant-VSA url-filtering lookup url mail.yahoo.com
lookup_result URL : mail.yahoo.com/
Pre-defined category count : 0
User-defined category count : 0
Pre-defined reputation,
    Index : 40, Name: suspicious
User-defined reputation,
    Index : 0, Name: undefined
NOTE: Predefined results are from spack database.  Spack flavor is Premium.  Predefined URLF database is loaded successfully.

(2) Here, we are manually instigating a cloudlookup with caching enabled.

admin@Branch-cli> request orgs org-services Tenant-VSA url-filtering cloud-lookup lookup url mail.yahoo.com cache-response true
clookup_result Status           : SUCCESS,
URL              : mail.yahoo.com,
Time taken       : 148 ms,
All-one-category : 0,
Reputation       : 96 (trustworthy),
Categories,
     ID :    55, Confidence:    93, Name : web_based_email
Response cached  : True,

(3) Now we re-try the GUI or a regular lookup, we would see a note saying "Predefined results are from cloud lookup cache."

admin@Branch-cli> request orgs org-services Tenant-VSA url-filtering lookup url mail.yahoo.com
lookup_result URL : mail.yahoo.com/
Pre-defined category count : 1
    ID: 55, Confidence: 93, Name: web_based_email
User-defined category count : 0
Pre-defined reputation,
    Index : 96, Name: trustworthy
User-defined reputation,
    Index : 0, Name: undefined
NOTE: Predefined results are from cloud lookup cache.  Spack flavor is Premium.  Predefined URLF database is loaded successfully.