This presentation summarizes a line of research that maps certain, notoriously hard, problems of discrete mathematics---such as the Hamiltonian Cycle and the Traveling Salesman Problems---into convex domains where continuum analysis can be carried out. Arguably, the inherent difficulty of these, now classical, problems stems precisely from the discrete nature of domains in which these problems are posed. The convexification of domains underpinning the reported results is achieved by assigning probabilistic interpretation to key elements of the original deterministic problems.

In particular, approaches summarized build on a technique that embeds Hamiltonian Cycle and Traveling Salesman Problems in a structured singularly perturbed Markov Decision Process. The unifying idea is to interpret subgraphs traced out by deterministic policies as extreme points of a convex polytope in a space filled with randomized policies. Special attention is devoted to the subset of the latter that corresponds to all doubly stochastic probability transition matrices induced by a graph. By the famous Birkhoff--von Neumann Theorem that subset is the convex hull of permutation matrices induced by the given graph. Clearly, Hamiltonian cycles (if any) are among the extreme points of this doubly stochastic polytope.

The topic has now evolved to the point where there are many, both theoretical and algorithmic, results that exploit the nexus between graph theoretic structures and both probabilistic and algebraic entities of related Markov chains. These include moments of first return times, limiting frequencies of visits to nodes, or the spectra of certain matrices traditionally associated with the analysis of Markov chains. There are also interesting connections with singular perturbation theory of mathematical programs and the cross-entropy method for the estimation of rare events. A number of open questions and problems will be described in the presentation. The latter will include a conjecture suggesting that instances of graphs where the underlying "NP-difficulty" of the Hamiltonian cycle problem is a substantive issue are exceptionally rare.