How to integrate LogDigger with Zendesk – quick tutorial

Every application sooner or later comes up against some problems. Some of them are not so obvious and are very hard to spot. But others are “screaming” to users and providers when they materialize. Today we will be talking about the second sort of them.

You noticed that your application encounters some issues. But how to protect your application? You could prepare appropriate design, do some automatic or manual tests and so on. Ok, these are good things to do before the release time, but what should you do after your application goes live into production environment? I am pretty sure that you have prepared good architecture with all stability and capacity patterns in mind, but what about crashes and some unexpected failures?

There are many tools and solutions that provide monitoring of applications. You could i.e. prepare some external tools like: live check by http request (web applications) or log file scraping by some CRON script. But one of my favourite is to prepare embedded mechanism, which follows user activities, but in the same time is fully transparent for them. You could build this kind of mechanism with LogDigger - tool that helps you to collect and classify detailed error reports and logs for your Java applications. More about LogDigger you can find on this page.

In this tutorial, I would like to show you how to create appropriate solution to collect your application error logs with LogDigger. Moreover I will describe how to send these error logs spotted by LogDigger to your Zendesk, so you can automatically manage your application bugs in an organized way.

Add LogDigger jars to your application

First you should Download LogDigger Connector and copy jar files to your application’s WEB-INF/lib.

Add LogDigger servlet filter to application’s web.xml

Next you should add some filter to track user activities and collect errors if happens.

See LogDigger Servlet Filter for instructions and configuration sample.

If you’re using Spring Framework, check this article Using LogDigger with Spring Framework.

If everything goes well, then the LogDigger Console will be accessible within your application at <YourAppURL>/logdigger/console

Test LogDigger’s error capturing capabilities

You don’t have to break your application in order to test LogDigger. Add the TestExceptionGenerator module to the LogDigger’s configuration string and follow usage instructions to generate a test error report.

Open the Console again to review the created error report and logged data.

Create own ErrorLogStore

Now goes the clue of this tutorial. In order to persist error logs and add additional functionalities to LogDigger, it is good practice to create your own ErrorLogStore. You could do this by creating class, which implements:

com.logdigger.connector.ErrorLogStore;

This class could provide DB persistence for your error logs, but you could add additional features to some methods. First I show you how to create persistence to your error logs. To do this, I use JdbcErrorLogStore provided by LogDigger and wrap its all methods.

public class ZendeskErrorLogStore implements ErrorLogStore {

    private JdbcErrorLogStore jdbcErrorLogStore;

    //errors to ommit before storing
    private String[][] ignoredErrors;

    //Mandrill API URL
    private WebResource webResource;

    //Mandrill API KEY
    private String mandrillApiKey;

    //LogDigger URL
    private String logdiggerAddress;

    @Override
    public void init(AttributeProvider attributeProvider) {
        jdbcErrorLogStore = new JdbcErrorLogStore();
        jdbcErrorLogStore.init(attributeProvider);

        ignoredErrors = new String[][] {
            {
                "com.someCompany.SomeException"
            }
        };

        mandrillApiKey = String.valueOf(attributeProvider.getAttribute("mandrillApiKey"));
        logdiggerAddress = String.valueOf(attributeProvider.getAttribute("logdiggerAddress"));
        webResource = Client.create().resource("https://mandrillapp.com/api/1.0/");
    }

    @Override
    public void destroy() {
        jdbcErrorLogStore.destroy();
    }

    @Override
    public ErrorLog getErrorLogById(String id) {
        return jdbcErrorLogStore.getErrorLogById(id);
    }

    @Override
    public ResultsPage<ErrorLog> findInDateRange(Date fromDate, Date toDate, int offset, int limit) {
        return jdbcErrorLogStore.findInDateRange(fromDate, toDate, offset, limit);
    }

    @Override
    public ResultsPage<ErrorLog> findSimilar(ErrorLog sample, int offset, int limit) {
        return jdbcErrorLogStore.findSimilar(sample, offset, limit);
    }

    @Override
    public ResultsPage<ErrorLog> getErrorLogs(int offset, int limit) {
        return jdbcErrorLogStore.getErrorLogs(offset, limit);
    }

    @Override
    public LastStoreError getLastStoreError() {
        return jdbcErrorLogStore.getLastStoreError();
    }

    @Override
    public int getMaxCategories() {
        return jdbcErrorLogStore.getMaxCategories();
    }

    @Override
    public String getStoreDescription() {
        return jdbcErrorLogStore.getStoreDescription() + " + ZenDesk extension";
    }

    @Override
    public boolean isAvailable(Feature feature) {
        return jdbcErrorLogStore.isAvailable(feature);
    }

I omit one very important method in listing above – save - because it deserves more attention.  In the example below you can see how to ignore some errors (your company or other vendor errors), how to store errors in DB and also (TADAM! ;) ) how to create issue in Zendesk with Mandrill API omiting duplicates. You could use JavaMail API instead of Mandrill API if you like.

  @Override
    public String save(ErrorLog errorLog) {
        if (errorLog.getCategorySignature(0).startsWith("com.mobeelizer.")) {

            // ignore all Mobeelizer exceptions
            return null;
        }

        //ignore all given exceptions
        for (String[] ignoredError : ignoredErrors) {
            for (int i = 0; i < ignoredError.length; i++) {
                if (ignoredError[i] == null) {
                    return null;
                }
                if (!ignoredError[i].equals(errorLog.getCategorySignature(i))) {
                    break;
                }
                if (i >= ignoredError.length - 1) {
                    return null;
                }
            }
        }

        //persist error log in DB
        String id = jdbcErrorLogStore.save(errorLog);

        //omit bug report if bug was reported
        if (jdbcErrorLogStore.findSimilar(errorLog, 0, 1).getTotalCount() == 1) {
            try {

                //create bug report content (stack + LogDigger console link)
                String content = (logdiggerAddress + "/console/view?err=" + id + "\n\n" + errorLog.getStackTrace()).replaceAll(
                        "[\n\t\r]+", "\\\\n");

                //create bug report subject
                String subject = "LogDigger " + errorLog.getLogSnippet().split("\n")[0];

                //Mandrill API to send mail
                WebResource path = webResource.path("messages/send.json");

                //create builder to send bug report
                Builder b = path.type(MediaType.APPLICATION_JSON_TYPE).accept(MediaType.APPLICATION_JSON_TYPE);

                //create Mandrill email message
                String s = ("{'key': '" + mandrillApiKey + "', 'message':{'text': '" + content + "', 'subject': '" + subject + "', 'from_email': 'no_reply@mobeelizer.com', 'to':[{'email': 'helpme@mobeelizer.com'}]}}")
                        .replaceAll("'", "\"");

                log.info(" logdigger: " + s);

                //report bug
                ClientResponse response = b.post(ClientResponse.class, s);
                if (response.getStatus() != 200) {
                    log.warn("Mail sending error - status: " + response.getStatus() + ", content: "
                            + IOUtils.toString(response.getEntityInputStream()));
                }
            } catch (IOException e) {
                log.warn(e.getMessage(), e);
            }
        }

        return id;
    }

You can see below how does such issue created automaticaly by LogDigger looks like in Zendesk.

Zendesk

Summary

We could state that monitoring deployed application in production environment is a very useful thing. It is always a better situation when monitoring bring some automation to create notifications (email notifications) or tasks (bug tracking system issue, help desk issue). The value of this automation is priceless, because it gives you time. You could get notification about problems before client see them (if observe), but for sure you get them before client issue report them to you. Additional very important element is that in issue/bug tracking system you always have few important information, which you have to look for them in logs making your client waiting and loosing patience on you :)

Above mechanism will not help your software pass QA or in production start guarantee more stable life for your solution but they can help you get a full night’s sleep, or an uninterrupted date with your girlfriend/wife, once your software launches. :)

This entry was posted in Dev and tagged , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>